This Is The One Key Difference Separating Good Science From Junk Science

The line between science and pseudoscience is imperceptible to most. Here’s how you can catch it in yourself.

When it comes to science, the first principle is that you must not fool yourself, and you are the easiest person to fool. This is true in all fields of science: from social sciences to physical sciences; from theory to experiment to observation. Whenever you’re considering a scientific problem, any well-meaning inquiry will attempt to formulate and determine a decisive solution to it, uncovering the secrets of the natural workings of the Universe.

For the scientist working towards that solution, though, there’s an extraordinary danger afoot. No matter how dispassionate we attempt to be, we cannot help but be colored by the biases of our preferred outcome, no matter how subtle or overt it is. If we perform our scientific inquiry with a view towards the results supporting the conclusion we desire, we run the risk of doing nothing but junk science, rather than sound science.

This can manifest itself in any number of ways. The most obvious is through outright scientific fraud, where the investigators begin with the conclusion they wish to reach and work backwards from there. If you’re someone who’s convinced of what the conclusion must be before you even begin, such as:

• that human beings were created in their current form by an all-powerful, all-knowing being,
• that the current changes Earth’s climate are experiencing must be due to the Sun,
• that the extinction of the dinosaurs could not be due to an asteroid strike,
• or that chemicals found in vaccines are the cause of autism in humans,

you’re going to find a way to validate that conclusion regardless of what the data says. Countless scientific frauds — involving everything from selection bias to the post hoc fallacy to simply faking your results — have been perpetrated in this manner upon both the scientific communities and general public.

But fraud is a strong word, and implies that there’s an intent to deceive. Oftentimes that isn’t the case at all; the researchers aren’t engaging in an elaborate scheme to deceive an external observer. Instead, they’re victims of what Richard Feynman was alluding to when he said, “and you are the easiest person to fool.”

Researchers of all types can succumb to self-deception, where you over-emphasize the portions of the evidence that support your preferred conclusion, while simultaneously discounting any evidence that points away from it. This is a classic way that motivated reasoning can manifest itself, which, as human beings, we do all the time in our daily lives.

When we disagree and argue and debate, the victor is usually the side with the more persuasive argument. In science, however, persuasion often has very little to do with scientific truths. In the early 1800s, there was a debate over whether light was a particle or wave, with a compelling argument demonstrating the absurdity of the wave theory put forth by Simeon Poisson, winning over many scientists. However, when the apparently absurd prediction was actually put to the test by François Arago, it was borne out, demonstrating that reductio ad absurdum does not work in science. Only experiments and observations can determine truths about the Universe.

This occurs again and again throughout the history of science, whenever conventional wisdom is overturned.

• Ludwig Boltzmann faced massive ridicule for his work on thermodynamics, eventually committing suicide in 1906. His work was largely carried on and extended by Paul Ehrenfest, who faced similar ridicule, committing suicide in 1933. Their work laid the foundation for modern statistical mechanics.
• In 1927, Georges Lemaître put together data about the redshift and distance measurements of galaxies to infer the expanding Universe, writing to Einstein about his findings. Einstein responded, “Your calculations are correct, but your physics are abominable.” Yet Lemaître was correct, with his conclusions predating Hubble’s identical ones by two years.
• Fritz Zwicky, who first inferred the existence of dark matter in the 1930s, had his results dismissed based on the absurdity that such a significant fraction of the Universe could be hitherto undetected. The work of Vera Rubin and Kent Ford in the 1970s led to dark matter being seriously considered, but the work of Zwicky could have given us a 40 year head-start on the puzzle.

The thing is, each of these scientists had a great, compelling, interesting idea, and went to work on it. But they also were guided by data and evidence that pushed them in that direction, rather than ideology alone. New ideas that fly in the face of the current scientific consensus may be the way that many scientific theories arise and the way that scientific revolutions occur.

But this is also how bad ideas arise, and more dangerously, how they persist. When you formulate a scientific theory that attempts to replace the existing one, you absolutely must come up with a set of tests that allow you to discern its predictions, robustly, from the currently dominant paradigm.

The way new scientific theories emerge and replace old ones isn’t set-in-stone, but there are three features that they all share. They are:

1. able to reproduce all of the successful explanations that previously-dominant theory explained,
2. able to successfully explain at least one already-observed or already-measured phenomenon that the prior theory couldn’t explain, and
3. able to make new and measurable predictions, for hitherto unobserved phenomena, that will be distinct from the predictions of the current theory.

This is how Einstein’s General Relativity came to replace Newton’s gravity. This is how plate tectonics rose to geological prominence. This is how DNA added an additional layer to genetics, which itself added an additional layer to Darwinian evolution. And this is how cosmic inflation advanced the Big Bang, which itself advanced our picture of the expanding Universe.

This is why it’s of paramount importance to make sure you aren’t fooling yourself. You can fit galaxy rotation curves without dark matter, by modifying the laws of gravity. You can explain the orbits of a large suite of trans-Neptunian objects by invoking a large planet many times the Neptune-Sun distance. And you can explain the masses of protons and neutrons without requiring the Higgs boson.

If these are the outcomes you desire, though, you must make sure to protect yourself against discounting the evidence that contradicts these conclusions. You must look at the full suite of evidence, honestly and robustly, and make sure you meet all of the necessary criteria. When you do, dark matter succeeds in a myriad of regimes were modified gravity fails; Planet Nine’s existence faces serious doubts; and the Higgs exists as the explanation for the rest masses of fundamental particles.

It’s extraordinarily tempting to imagine that the next great advance in science is just a breath away, and even more tempting to believe that you, yourself, are going to be the one who discovers it. That’s why you should be extra skeptical of grandiose claims when they’re made by a person who’s spent a considerable fraction of their life promoting one particular alternative to the mainstream. Science requires independent checks and verification of experiments; science requires the gathering of more and better data to strengthen and further validate preliminary conclusions; science requires that theories have predictive power that can be robustly tested.

You must not look at only the data that supports your conclusion; you must examine and consider the full suite of data. We might sometimes fool ourselves, but sound, reproducible science will always carry the day. As Feynman said, “reality must take precedence over public relations, for Nature cannot be fooled.”