A simple principle that explains everything from the perceived success of speed cameras and alternative medicine to the Sports Illustrated jinx

According to urban legend, sports stars who appear on the cover of Sports Illustrated will subsequently experience bad luck. This seems counterintuitive but time and time again the Sports Illustrated jinx repeats itself. Take another example that at first might seem unrelated: Flight cadets have been noted to improve in performance immediately following punishment and show poorer performance immediately following rewards – a finding that seemingly flies in the face of much of what we know about behavioural psychology. Think about your own life for a moment, have you ever noticed this effect? Why might this be the case?

In both cases, as Nobel prize winning economist Daniel Kahneman explains in Thinking Fast and Slow, the answer is nothing to do with what we might expect – the fact that the sportsmen and women appeared on Sports Illustrated or the way the flight cadets were rewarded or punished. The real explanation is all down to probability – a principle known as regression to the mean. Wherever we can expect random fluctuations, if we take an extreme example such as a sportsmen who is on a winning streak or a flight cadet performing particularly well or badly, we can expect that in the immediate future their performance will return closer to the mean due to chance alone. A classic example of this is in how we interpret the effect of speed cameras which are typically installed following a streak of accidents. Following a streak of accidents we can expect that due to random chance there will be a period of time where no accidents take place, regardless of whether or not a speed camera is installed – but when a speed camera is installed we attribute the subsequent immediate drop in accidents as a result of the speed camera.

Regression to the mean can be used to explain the success of alternative medicine (i.e. any intervention that has not passed the test of randomised controlled trials). This is due to the fact that people often tend to get better by themselves given time – they regress to the mean. Statistically, having a headache for example, is an extreme situation. You’re most likely to seek help for symptoms when the pain is at its most severe. If you have a headache and take a homeopathic pill you might feel better because of the placebo effect – but you might just feel better because your headache passed all by itself – regression to the mean. This can be a problem for all kinds of research if an experiment is not carefully controlled. If a patient group is selected for their extreme symptoms we can expect that as a group their condition will generally improve over time regardless of any intervention.

A thought experiment presented by Daniel Kahneman is to consider the case that “highly intelligent women tend to marry men that are less intelligent than they are”. Take a moment to think about that. As humans, we naturally tend to spot patterns such as this and come up with our own causal explanations. The solution proposed by Kahneman is somewhat more straightforward: “If the correlation between the intelligence of spouses is less than perfect and if men and women on average do not differ in intelligence then it is a mathematical inevitability that highly intelligent women will be married to husbands who are on average less intelligent than they are – and vice versa of course”. Whether we like it or not we are programmed to seek causal explanations even when they do not exist and that is something we need to actively remember if we don’t want to be misled – or mislead ourselves.

It’s staggering how few adults understand the concept of regression to the mean, but it really is a concept worth getting to grips with. This video introduces the concept beautifully:

To keep up to date with this blog you can follow Neurobonkers on Twitter, Facebook, RSS or join the mailing list.

Image Credit: Shutterstock/soliman design