A Response to Science on the Decision to Not Include Evolution in the NSB Science Indicators Report

At Science today, contributing journalist Yudhijit Bhattacharjee reports on the decision by the National Science Board to drop discussion of survey questions about evolution from their 2010 Science Indicators report. As a reviewer of several previous versions of the report and as an expert who provided input and feedback on the design of the 2006 survey instrument, I have several thoughts on what I think Bhattacharjee in the article unfairly portrays as a “controversy.”

The NSB is correct to be concerned about how these questions are interpreted by the public and by the scientific community, but by not including a discussion of these indicators in the report, the NSB missed an important opportunity to educate the science community about the nature of public opinion and the many social factors that shape public perceptions and understanding.

The NSB’s decision was based on the accurate premise that the evolution question taps for the most part religious beliefs rather than factual knowledge. I’ve pointed to the validity issues surrounding this question in talks over the past few years including most recently in a lecture I gave last year at the National Academies (see discussion starting at 24 minute mark of video).

Consider what a split-ballot comparison in a 2004 University of Michigan survey revealed about the nature of responses to these long standing questions about evolution. In this survey experiment, one half of the sample was asked the following traditionally worded question:

True or false, human beings as we know them today, developed from earlier species of animals.

When asked this way, 42% answered true, a result that has been incredibly consistent across surveys since 1985.

The other half of the sample, however, was asked a slightly different version of the question:

True or false, according to the theory of evolution, human beings as we know them today, developed from earlier species of animals.

When asked this way, 74% answered true.

The implication is that context matters: Americans are not ignorant of what science says about human origins, in fact, as the second version of the question reveals, 3/4 of the public are familiar with the scientifically correct answer.

But when presented with the traditional version of the question, Americans are asked to choose between what they know to be the scientifically correct answer and their own religious beliefs. Therefore, as a direct measure of scientific knowledge, unlike the other items included in the scale measuring science literacy, the evolution item scores low in terms of validity.

As I wrote with my colleague Dietram Scheufele last year in a review article, the NSF Science Indicators reports have been used rhetorically by science advocates and pundits to reinforce a false deficit model about the public, one that decries widespread ignorance and that promotes a constant “science under siege” mentality. This outlook is distracting and harmful to public engagement.

It’s also a myth. While Americans might score low on quiz like questions about technical areas of science, other questions reveal that Americans believe strongly in the promise of science and that scientists and their institutions hold almost unrivaled levels of public trust. Moreover, despite what many scientists and advocates might describe as an “enlightened” European public, the U.S. public scores as well if not better than many European publics when it comes to overall science literacy. The one indicator that falls below scores in other countries is the question about evolution. The NSB was therefore wise to be cautious about the reporting of this question.

Still, however, despite how the survey results have been misused and misinterpreted in the past, the NSB should have included a careful and detailed discussion of the evolution results in the chapter. I served as a reviewer on the 2006 and 2008 versions of the chapter but was not asked to review the 2010 version. My advice as a reviewer would have been to highlight in a call out box this problem with the survey question and in the process to emphasize that when it comes to public understanding of evolutionary science, the public is not ignorant, but instead divided in choosing an answer between what they know to be scientifically true and what they believe in terms of religious faith.

Including this type of context and explanation in the report would have gone a long way to improving the literacy of the science community, journalists, and policymakers about the complexities of science-society relations. This I hope will be a central goal of future chapters.

UPDATE: At the blog Thoughts from Kansas, Josh Rosenau, a staffer at the National Center for Science Education, and who is quoted in the Science article, links to a draft version of the chapter (PDF) that was uploaded at the Science web site.

Looking at the draft version, it does a very good job of explaining what I outlined above, adding to the Michigan survey by drawing on findings from Gallup. Why this section was deleted–given its value in shedding light on the complexity of survey response and public views on the issue–is not clear. Again, if I were a reviewer, I would have argued strongly for its inclusion, making it a highlight of the report. See specific text below:

Americans’ responses to questions about evolution and the big bang appear to reflect factors beyond familiarity with basic elements of science. An experiment conducted in the 2004 Michigan Survey of Consumer Attitudes showed that respondents were more likely to answer these two questions correctly when the questions were prefaced by ―according to the theory of evolution or ―according to astronomers. These differences probably indicate that many Americans hold religious beliefs that cause them to be skeptical of established scientific ideas, even when they have some basic familiarity with those ideas (for additional details see NSB 2008).

Recent surveys conducted by the Gallup Organization provide similar evidence. A 2009 survey showed that more than half (55%) of Americans could correctly name evolution or another closely associated term, such as natural selection, when asked which scientific theory they associate with Charles Darwin. However, in a follow-up question, only 39% of Americans say they believe in the theory of evolution, 25% say they do not believe in this theory, and 36% do not have an opinion on this subject either way (Newport 2009).