10 Scientific “Breakthroughs” That Don’t Stand Up to Scientific Scrutiny
A crucial element of scientific progress is the reproducibility of experiments and their results. A problem in contemporary academic institutions has been the discouragement of repeating experiments. The Economist reported in 2013 that in the midst of this deep lack of support for reproducing published studies, some people have been pushing back – most notably in the field of psychology.
A recent article by Christian Jarrett in The British Psychological Society’s Research Digest reveals both how active some researchers have been in scrutinizing influential findings through replicated trials from and how deep the stakes are for encouraging such efforts. He sheds light on ten common claims in psychology that are often cited and invoked as true but have not held up to re-testing:
1. Confident body language makes people feel more confident.
2. Smiling makes people feel happier.
3. Individuals have a finite amount of self-control, where “the more of it you use in one situation, the less you have left over to deal with other demands.”
4. Making revisions after taking an exam can retroactively help to improve scores. [This seemingly unlikely yet influential finding now, like the others on this list, seems unlikely to be true.]
5. Behavior can be influenced by exposure to certain kinds of words. More specifically, “Exposure to words pertaining to aging will make you walk more slowly.”
6. Hygienic cleaning helps to ameliorate moral guilt (aka the “Macbeth Effect”).
7. Babies imitate others’ behaviors.
8. Images and feelings of being watched compel people to act more honestly.
9. Smelling oxytocin, the neurochemical released during cuddling and sex, makes people more trusting of others.
10. Mentioning money makes people act more selfishly.
Image: nature.com
Jarrett locates the experimental origins of all of these influential, frequently cited claims. Then, with each one, he reveals a battery of recent tests that have been done aiming to replicate the findings. In every case, the attempts have been unsuccessful, usually indicating no relationship between the relevant variables (e.g., between smiling and feelings of happiness).
Yet the paper arguing that smiling makes the smiler feel more pleasant has been cited more than 1,500 times. Meanwhile, its conclusions continue to elude researchers trying to reproduce the experimental results or scrutinize the original data. Jarrett points to a recent study in Psychological Science critiques the original paper about bold body language as well as a follow-up that was published and finds that the experimental data that was used does not reflect statistically significant data. Professors Joseph P. Simmons and Uri Simonsohn, the authors of the recent critiques, sum their findings succinctly by asking: “Does the literature reviewed by Carney et al. (2015) [the researchers behind the original paper] suggest the existence of an effect once we account for selective reporting? We conclude not.” They arrive at their conclusion through statistical analysis of the original data.
The purported happiness-inducing effects of smiling have similarly not been standing up to scrutiny. As Jarrett demonstrates in another article in Research Digest, 17 different labs tried independently to regenerate this well-known finding. And they failed. The researchers claimed, “Overall, the results were inconsistent with the original result.” Not only do these findings suggest that the original finding is difficult to generate but also that, given how many studies have now been done, that the original conclusions may most likely be deeply wrong.
The failure of so many findings in academic and popular psychology to persist through repeated trials has direct implications for how we ought to apply the findings to our lives. For example, if confident body language does not evoke feelings of confidence, then gesturing boldly when feeling insecure may be effective only in appearing more awkward.
More broadly, these studies demonstrate the importance of actively encouraging researchers to perform and publish on repeat-trials. This, as Jarrett demonstrates, is crucial to interrogating consequential conclusions of experimental researchers. The stakes are high. If the causal link between smiling and a spike is happiness is blurrier than we once thought, what other claims are taken for granted in psychology? In physics? In medicine?
