Why the number 2.9013 will go down in the history of bad science
If you’ve been reading this blog for any amount of time you’ll probably be familiar with the name Sokal from the Sokal affair, the scandal in 1996 in which physicist Alan Sokal intentionally submitted a paper that was a steaming pile of horse manure in a leading journal of cultural studies, designed as a test to see if it would get published (spolier, it did).
At the time, Sokal came under harsh criticism from some quarters for creating a “tempest in a tea cup” and for making “tendentious misrepresentations”, according to some he had “insufficiently grasped the philosophy (he) criticized”. Well the tables have turned and now Sokal along with Nicholas Brown and Harris Friedman have published a paper that mercilessly destroys a paper on positive psychology that has been cited close to a thousand times on Google Scholar and in countless self help books, somehow with no one in the field realizing the paper was not worth the paper it was written on.
The original paper by Fredrickson & Losada makes the case that the “positivity ratio” – the relationship between an individual’s positive and negative emotions reaches a critical tipping point when it reaches the number 2.9013.
But as Sokal et al make painfully clear, the paper is utterly brimming with “fundamental conceptual and mathematical errors” and “the total absence of any justification for the applicability of the Lorenz equations to modeling the time evolution of human emotions”. With me until that last bit? Well the original paper stated that the “positivity ratio” followed the following, completely irrelevant complex mathematic pattern:
Sokal et al point out that “the idea that any aspect of human behavior or experience should be universally and reproducibly constant to five significant digits would, if proven, constitute a unique moment in the history of the social sciences”. This before demonstrating the fatal mathematical and logical flaws at every level, before concluding “that Fredrickson and Losada’s (2005) claims concerning the alleged critical values of the positivity ratio are entirely unfounded… One can only marvel at the astonishing coincidence that human emotions should turn out to be governed by exactly the same set of equations that were derived in a celebrated article several decades ago as a deliberately simplified model of convection in fluids, and whose solutions happen to have visually appealing properties”.
The final verdict is damning and enlightening:
“An alternative explanation—and, frankly, the one that appears most plausible to us—is that the entire process of “derivation” of the Lorenz equations has been contrived to demonstrate an imagined fit between some rather limited empirical data and the scientifically impressive world of nonlinear dynamics.”
The crack team point out the mind bending absurdness of the conclusions Fredrickson and Losada abstract from their unbelievable (in the truest sense of the word) findings:
“They appear to assert that the predictive use of differential equations abstracted from a domain of the natural sciences to describe human interactions can be justified on the basis of the linguistic similarity between elements of the technical vocabulary of that scientific domain and the adjectives used metaphorically by a particular observer to describe those human interactions. If true, this would have remarkable implications for the social sciences. One could describe a team’s interactions as “sparky” and confidently predict that their emotions would be subject to the same laws that govern the dielectric breakdown of air under the influence of an electric field. Alternatively, the interactions of a team of researchers whose journal articles are characterized by “smoke and mirrors” could be modeled using the physics of airborne particulate combustion residues, combined in some way with classical optics.”
Sokal et al don’t postulate with regard to whether the cause of the obviously unlikely finding is the result of “excessive enthusiasm, sincere self-deception, or other motivations”.
In a response from Fredrickson (her coauthor has gone mysteriously AWOL), she concedes that “Losada’s mathematical work may have been flawed” – and endeavors to scrub this part of the theory from the record, leaving the original psychological theory intact (which in fairness it may well be, this mumbo jumbo aside).
This all begs the question, how much more of the literature is steeped in fluffy feathered nonsense? “Filigrees of rhetorical precision atop unsteady pillars of conceptual bluff”. The answer is potentially a hefty chunk.
I stumbled on one particularly priceless gem myself only last week, disasters this big are thankfully pretty rare in science, but it just goes to show how important it is to always read critically yourself, even if a thousand other researchers in your field have given something the green light, it doesn’t mean they are right. The whole fiasco reminds me of the bogus TedX talk which received a standing ovation, months went by before it was discovered that the lecture on how “vortel based maths” would save the world was complete and utter unmitigated nonsense.
If anything, my guess would be that this is an example of groupthink, did the researchers that cited this paper really believe it, did they even read it? Or did they see that other esteemed psychologists had cited it and think… “X cited it… must be legit”. It’s something we’re all guilty of doing on some level at some point or another, if we insisted on checking every single “fact” we came across in life ourselves, we’d be both hopelessly ill equipt and have a hard time functioning in society. But when making a scientific argument, or analyzing the particularly earth shattering ones, it’s a practice clearly worth taking up.
Another thing worth pondering is whether this study was so widely cited simply because at first glance it appears more “sciencey” than traditional positive psychology research. In the aftermath of The Seductive Allure of the “Seductive Allure” of Neuroscience Explanations debate, this couldn’t have been more timely.
I’m going to stop here to avoid simply rehashing what has already been said more eloquently elsewhere. For further analysis, I’ll pass you over to the venerable Neuroskeptic over at Discover and Will Willkinson over at the Daily Beast.
References
Brown N.J.L., Sokal A.D. & Friedman H.L. (2013). The Complex Dynamics of Wishful Thinking: The Critical Positivity Ratio., The American psychologist, PMID: 23855896 (PDF)
Fredrickson, B. L., & Losada, M. F. (2005). Positive affect and the complex dynamics of human flourishing. American Psychologist, 60, 678–686. doi:10.1037/0003-066X.60.7.678 (PDF)
Fredrickson B.L. (2013). Updated Thinking on Positivity Ratios., American Psychologist, DOI: 10.1037/a0033584 (PDF)
To keep up to date with this blog you can follow Neurobonkers on Twitter, Facebook, RSS or join the mailing list.
Image Credit: Shutterstock/Wikimedia Commons
Update: This article was amended on 23/08/13 to include the line “At the time, Sokal came under harsh criticism from some quarters for creating a “tempest in a tea cup” and for making “tendentious misrepresentations”, according to some he had “insufficiently grasped the philosophy (he) criticized” – so that the following remark makes sense in context.
