Based on the book 'The Demon Haunted World' by Carl Sagan, the following are suggested as tools for testing arguments and detecting fallacious or fraudulent arguments:
- Wherever possible there must be independent confirmation of the facts
- Encourage substantive debate on the evidence by knowledgeable proponents of all points of view.
- Arguments from authority carry little weight (in science there are no “authorities”).
- Spin more than one hypothesis - don’t simply run with the first idea that caught your fancy.
- Try not to get overly attached to a hypothesis just because it’s yours.
- Quantify, wherever possible.
- If there is a chain of argument every link in the chain must work.
- “Occam’s razor” - if there are two hypothesis that explain the data equally well choose the simpler.
- Ask whether the hypothesis can, at least in principle, be falsified (shown to be false by some unambiguous test). In other words, it is testable? Can others duplicate the experiment and get the same result?
Additional issues include
- Conduct control experiments - especially “double blind” experiments where the person taking measurements is not aware of the test and control subjects.
- Check for confounding factors - separate the variables.