Was it all just noise? Independent analysis casts doubt on LIGO’s detections

The smart money is still on LIGO, but there sure looks like something funny is happening if the new group is correct.

This article was contributed by Sabine Hossenfelder. Sabine is a theoretical physicist specialized in quantum gravity and high energy physics. She also freelance writes about science.

“What’s really exciting is what comes next. I think we’re opening a window on the universe — a window of gravitational wave astronomy.” 
–Dave Reitze, executive director of LIGO

After an effort of more than 100 years and a collaboration involving over 1,000 scientists, we all celebrated. It was February 11, 2016, and LIGO had just announced their first direct detection of gravitational waves. Analysis of the data attributed the signal to a black hole merger that happened several billion light years away. But what if there wasn’t a signal at all, but rather patterns and correlations in the noise that fooled us into believing we were seeing something that wasn’t real? A group of Danish researchers just submitted a paper arguing that the celebration might have been premature.

A team of five researchers — James Creswell, Sebastian von Hausegger, Andrew D. Jackson, Hao Liu, and Pavel Naselsky — from the Niels Bohr Institute in Copenhagen, presented their own analysis of the openly available LIGO data. And, unlike the LIGO collaboration itself, they come to a disturbing conclusion: that these gravitational waves might not be signals at all, but rather patterns in the noise that have hoodwinked even the best scientists working on this puzzle.

The LIGO gravitational wave observatory consists of two experimental sites — one in Livingston, Louisiana and one in Hanford, Washington — each of which is a laser interferometer with arms that are several kilometers in length. Even for these super-sensitive detectors, however, gravitational waves are difficult to measure. The problem isn’t so much the absolute weakness of the waves, the problem is that there are many other disturbances that also wiggle the interferometer. The challenge, thus, is to tell the signal from the noise.

To identify a gravitational wave signal, LIGO relies on the combined signal from both detectors. A gravitational wave will travel through each site at a different time, since the signal travels at the speed of light, but the two sites are separated by thousands of kilometers. The signals that arrive should be correlated, but with a characteristic time-lag and an amplitude offset, since they’re oriented differently in space (on the surface of the curved Earth) in three dimensions. However, there shouldn’t be any such correlation in the noise. At least, that’s the idea.

The Danish group found, however, that the noise at both detector sites — and puzzlingly, between the two supposedly independent detectors — is also correlated. And worse, the correlation time is similar to the time-lag between the recorded signals, for each of the three so-far confirmed events. According to Andrew Jackson, the leader of the Danish group,

“If the correlation properties of signal and the noise are similar, how is one to know precisely what is signal and what is noise?”

That’s a really important realization. A correlation in the noise would not affect the individual signals at each of the sites. But in order to achieve a highly significant signal between the detectors, the LIGO collaboration takes into account how both signals are correlated. If this correlation were not reliable, because (for example) there was the possibility that noise correlations contaminated their data, the statistical significance of the detection would be reduced. In other words, what appears to be a signal might actually be caused merely by fluctuations. How much the statistical significance would be affected, however, the Danish researchers have not quantified.

To get a visual impression of how noisy the data is, look at the above figure. This plot shows the measurement in a 32 second window around the first LIGO detection. The red curves are for the experimental site in Livingston, the black ones for Hansford. The outer curves with the large amplitudes are the raw data. The inner two curves are the cleaned data — enlarged by a factor of 100 so you can even see it! The little bump at about 16 seconds is the event.

A few weeks ago, Andrew Jackson presented his results in Munich. A member of the local physics faculty (who’d rather not be named) finds the results “quite disturbing” and hopes that the collaboration will take the criticism of the Danes to heart. “Until LIGO will provide clear scientific(!) explanation why these findings are wrong, I would say the result of the paper to some extent invalidates the reliability of the LIGO discovery.”

Jackson is no unknown to the LIGO collaboration. Upon my inquiry with a member of the LIGO collaboration what to make of the paper, I got the annoyed reply that the collaboration’s management recommends to “respectfully respond that we have talked at some length with the group in the past and do not agree on the methods being used and thus with the conclusions.” Another let me know that a response is not planned.

A major shortcoming of the Danish group’s analysis that they pointed out to me is that the Danes use methods based on tutorials from the LIGO Website, but these methods do not reach the quality standard of the — more intricate — data analysis that was used to obtain the published results.

A specific detail that might explain the finding is that the LIGO strain data has a random drift, which is slow, but large compared to the noise itself. Cutting out part of the signal — as in the 32 second window displayed above — and Fourier-analyzing it then bears a risk of surfacing artificial peaks in higher harmonics of the time-window. This artifact can be remedied by smoothly fading out the ends of the interval, something that was either not done or not mentioned in the group’s criticism. This might be a possible reason for the correlation they find.

Andrew Jackson himself would be happy to be shown wrong. He hopes that “our concerns will be taken constructively and lead both to improved methods for data analysis and to a better understanding of LIGO’s results and its exciting long-term potential.”

Making sense of somebody else’s data is tricky, as I can confirm from my own experience. Therefore, I think it is likely the Danish group made a mistake. Nevertheless, I would like to see a clear-cut explanation and “they did something wrong” is too vague for my comfort. This is a Nobel-worthy discovery and much is at stake. Even the smallest doubt that something is at odds should be erased.