“Natural experiments” win the 2021 Nobel Prize in economics

Correlation implies causation, but correlation alone is not sufficient to justify a conclusion of causation — otherwise, we would have to blame Nicolas Cage movies for swimming pool drownings. Unfortunately, in all but a few fields, teasing apart correlation from causation is notoriously difficult.

One of the few exceptions is biomedical science. In a randomized experiment like a clinical trial, subjects are randomly assigned (an absolutely crucial step) to either a control group or a test group. The control group usually gets a fake pill that does nothing (a placebo), while the test group gets an actual drug. This design allows researchers to determine if the medicine works and what side effects it brings.

This design, however successful, is not possible or ethical in many circumstances. For example, it is not possible to randomly assign several nations to a certain economic policy and another group of nations to a different policy. Likewise, it is neither possible nor ethical to force-feed several thousand people bacon to see if they develop cancer. Social sciences, therefore, are often stuck with inferior methods.

But just because these methods are not as powerful as randomized experiments doesn’t mean that they are useless. Very interesting and often persuasive data can be extracted from well-designed research. Serendipitously, conditions in the real world sometimes mimic those of an experiment. Known as “natural experiments,” these situations arise when something resembling a control group and a test group naturally emerge, allowing researchers to compare the data between them. Social scientists, including economists and sociologists, and even some members of the harder sciences like epidemiologists often make use of natural experiments.

One of the most famous examples is that of the Victorian physician John Snow, who determined the rate of cholera deaths that occurred in households that received water from one of two different water companies. This was not planned; Snow had no way of randomly assigning houses to different companies. But the conditions of this “natural experiment” were good enough. Because of it, he was able to determine that one of the companies was far more likely to poison its customers with filthy water.

That brings us to the 2021 Nobel Prize in Economics, which went to three researchers in the U.S. who used natural experiments to great effect.

Natural experiments

On Monday, it was announced that David Card won the prize for “empirical contributions to labor economics,” while Joshua D. Angrist and Guido W. Imbens won for “methodological contributions to the analysis of causal relationships.”

Card’s work in the 1990s used natural experiments to contribute to big debates — still ongoing today — about the minimum wage and education policy. In a 1993 paper, Card and Alan Krueger examined the effect of raising the minimum wage on employment. New Jersey had raised its minimum wage but neighboring Pennsylvania did not, which created a natural experiment. Card and Krueger then compared fast-food joints in eastern Pennsylvania (“control” group) with those in New Jersey (“experimental” group). 

Subverting common wisdom, they found no evidence that employment fell in New Jersey following the increase in the minimum wage. Other potential unintended consequences — such as fewer new restaurants, cuts to benefits, or a shift from full-time to part-time work — did not occur. 

In a 1996 paper, Card and Krueger looked at the natural experiment created by differing education policies in the Carolinas back in the Jim Crow era. South Carolina was far more aggressive than North Carolina in moving resources away from schools for Black students toward schools for White students. This allowed for a straightforward comparison between the two states.

The results clearly showed that better funded schools and smaller class sizes led to higher wages later in life. Specifically, Black students in North Carolina did better than their peers in South Carolina, while White South Carolinians did better than White North Carolinians. Of similar importance, the study found that the differences in wages between these groups slowly eroded over time as the education gap began to narrow after the Jim Crow era ended.

While Card and Krueger’s studies were robust, their findings were not entirely accepted due to the friction that existed between economic theory and reality. As time went on, however, the simple methodologies Card and Krueger used were emulated in subsequent economic research. That is where the work of Angrist and Imbens comes in. 

A new way to determine causation

To use the example of the Nobel Committee, imagine that one company decides to give its workers bicycles for Christmas while another does not. This is a natural experiment with a test group and a control group, and it has lots of things that can be measured, like how many people from each company end up riding a bike to work.

Subscribe for counterintuitive, surprising, and impactful stories delivered to your inbox every Thursday

However, as a natural experiment, it can be difficult to determine how an individual person is affected by having received a bicycle. After all, they might choose not to use it, or perhaps they already were an avid biker. These variables complicate causal effects, such as the impact of bicycles on employees’ health.

This is where a tool proposed by Angrist and Imbens in a 1994 study, the local average treatment effect (LATE), becomes useful. It is possible to put the people from both companies into one of four groups:

1. Compilers (who will use the bicycle if they are given one);
2. Always Takers (who always will use a bicycle even if they weren’t given one);
3. Never Takers (who never will use a bicycle even if given one); and
4. Defiers (who will use their own bicycle if not given one but will refuse to use a bicycle if given one).

LATE lets us put the latter three groups to the side and examine only the compilers. Then, using a lot of math, the average effect of the “treatment” — in this case, being given a bicycle — can be determined for a member of that group. While it is not possible to determine the exact impact on any single person, details about the overall group can be extracted, such as the average health boost attributable to the employer-provided bicycle.

Taken together, the work of these economists has shed light on how the world actually works (rather than how it is supposed to work in theory) and how we can use empirical data to inform our economic decision-making. Importantly, this approach has been used to study a wide variety of subjects outside of economics, including the COVID pandemic.