Have You Fallen Victim to the Guru Effect?
Four years ago a paper by Dan Sperber published in the Review of Philosophy and Psychology coined the term: The Guru Effect – the tendency for people to “judge profound what they have failed to grasp.” The paper examines how self-professed Gurus have a knack for inspiring devotion through speaking in a way that confers profound understanding but in reality fails to deliver anything of actual substance.
Unnecessary use of obfuscating language is certainly not the preserve of complete charlatans. Academics are often guilty of inadvertently introducing new levels of obscurity to what they are communicating, through the use of language that is impenetrable to someone working outside their field. Last year Professor of Social Sciences Michael Billig, (most well-known for his involvement in the minimal group experiments) published Learn to Write Badly: How to Succeed in the Social Sciences, a blistering critique of some of the pillars of his own field.
One of Billig’s flagship examples is of the use of the term “ideational metafunction” where the simple word “content” would suffice. Billig proposes that in a wide range of cases, new words are unnecessarily created to artificially produce a “seal of technicality” to “demonstrate that we are professional social scientists, rather than ordinary people who happen to have wandered in from the rain.” Billig goes further, arguing that social scientists are dividing their fields of study by creating cliques that unnecessarily use different specialized vocabularies to describe the same things. Billig argues that this unnecessarily prevents social scientists (let alone the public) from understanding the work of other social scientists working in different areas and consequently creates fault lines between these groups:
“We will be training our postgraduates to react like readers of popular right-wing newspapers: they will learn what they don’t like without having to encounter personally the things or people that they so dislike.”
Two decades ago, professor of mathematics Alan Sokal published a paper titled “Transgressing the Boundaries: Towards a Transformative Hermeneutics of Quantum Gravity” in a special issue of the journal Social Text, that was supposed to be devoted to rebutting criticisms of post modernism and social constructivism. The parody paper, which read as a mishmash of complete nonsense, in fact consisted largely of genuine quotes from renowned intellectuals. Sokal followed the parody with the book “Fashionable Nonsense: Postmodern Intellectuals’ Abuse of Science” which exposed the abuse of terms from maths and physics to create complex sounding statements that upon closer analysis were in fact irrelevant and meaningless by leading academics in the fields of social science, psychoanalysis and philosophy. The incident which came to become known as the Sokal affair, shook the foundations of these disciplines – but it seems the lasting impact has been minimal. Today, Billig’s book makes a similar argument and the conclusion is all the more damning, coming not from an outsider, but from someone who has spent their life residing at the heart of the social sciences.
Billig goes beyond the abuse of big, impressive-sounding words, demonstrating that what has become the standard form of reporting for findings in the social sciences enables researchers to disguise the true nature of their experimental findings and exaggerate their results. According to Billig, the abuse of statistics to hide what is really happening in experiments has become not just routine, but the norm – a norm that is so ingrained that adhering to it is the only way that scientists publishing in the social sciences can expect to see their work in print:
“If you take the trouble, you can notice something unexpected. These advocates of the scientific approach routinely conceal important aspects of their experimental findings and just as routinely they exaggerate their results. They exaggerate and they conceal so regularly that they cannot be aware of what exactly they are actually doing, as they follow supposedly scientific procedures.”
By publishing little more in terms of data than statistical significance scores, scientists working in the social sciences are able to withhold information of fundamental importance from the reader:
“There is one crucial piece of evidence lacking in most experimental social psychology reports today: frequency scores. Although writing about the significant effects of particular variables, the authors typically do not say exactly how many of the participants might have been affected by the variable in question. As we shall see, when statistically significant differences are found, the reports usually imply that all participants were affected. That implication has to be a fiction”.
Billig presents a range of cases from the forefront of the social sciences where failure to report raw experimental results combined with the use of language that is vague beyond belief creates a double whammy resulting in published research that ends up telling us precious little about anything at all. An example of good science cited by Billig is Milgram’s famous electric shock experiments where the results were reported as frequency scores enabling Milgram’s striking results to be obvious for all to see. Billig laments that Milgram would probably not be able to publish his work today the way he did then:
“The journal editors would probably tell him to go back and analyse his result properly – and, as we shall see, that would mean making it more difficult to see what happened in his experiments.”
Billig contrasts Milgram’s obedience work with the more recent, but similarly shocking findings in the field of behavioural priming, findings which have been described as “Milgram on steroids“. Billig looked at findings that apparently showed people primed with rude words would interrupt the experimenter’s conversation if given the opportunity and that people primed with words associated with old age would walk more slowly down a hall way. The findings were the centre of the recent failed replication furore which shook the field of social psychology. Billig reverse engineered a sample of priming data taken from the rude words experiment, noting “it is only because something went wrong in one of the experiments that we are able to come close to getting a glimpse of the frequencies”. When Billig exposes the estimated frequency data, the priming findings suddenly become decidedly less spectacular – according to Billig’s estimate only three out of thirteen participants were actually affected by the variable – information that is completely unavailable in the published paper:
“Once the results are expressed in terms of people, rather than mean group differences, they look considerably less dramatic.”
Billig’s offensive against poorly communicated research doesn’t pull any punches and in keeping with this spirit he does not exclude himself from the firing line, wholeheartedly admitting his own work contains the very same problems detailed in his book:
“Just in case anyone is thinking of trawling through my early writings, in order to see whether I was guilty of the very faults that I am now accusing others of committing, let me spare them the bother. I was. In those days I was a good boy, still trying to do what I had been taught to do.”
Billig is not hopeful for change, writing: “I fear that I can recommend until I am fit to burst, but nothing much will change. My recommendations, whatever they are, will just be whispers in the wind.” Billig is certainly right about the scale of the problem and its institutionalised nature – it is difficult for social scientists to publish in plain English and be taken seriously enough to have their work accepted for publication in leading journals; but Billig makes a strong case that a move towards clearer communication would be a good thing for the social sciences. Anyone who is engaged in the social sciences who has been frustrated by the opaque nature of the standard of communication would do well to read his book.
Image Credit: Shutterstock.
Follow Neurobonkers on Twitter, Facebook, RSS or join the mailing list.
