Polls relating to publicly controversial scientific issues often trigger a great wailing and gnashing of teeth from science advocates. When large proportions of a population seem poorly informed about evolution, climate change, or genetically modified foods, the usual response is to bemoan the state of science literacy. It can seem obvious that many people don’t understand the science of evolution, for example—or the scientific method, generally—and that opinions would change if only we could educate them.

Research has shown, unfortunately, it's not that simple. Ars has previously covered Yale Professor Dan Kahan’s research into what he calls “cultural cognition,” and the idea goes like this: public opinion on these topics is fundamentally tied to cultural identities rather than assessment of scientific evidence. In other words, rather than evaluate the science, people form opinions based on what they think people with a similar background believe.

That shouldn’t come as a shock, especially given the well-known political or religious divides apparent for climate change and evolution.

A key feature of Kahan's work, though, comes when he measures general science literacy or propensity for analytical thinking. Rather than ameliorating differences on scientific issues, these properties exacerbate them. Those who should be best equipped to have a handle on the science are the most divided along party lines. It seems that people more familiar with science are better at coming up with explanations to defend whatever conclusions their cultural group has reached.

All this means that when people respond to surveys asking whether they think Earth’s climate has warmed, for example, their answers tell you more about their cultural identity than their factual knowledge. To put that another way, general science knowledge is a very poor predictor of whether people think the Earth has warmed, but political party affiliation is a pretty good one.

Of course, general science knowledge (like knowing that electrons are smaller than atoms or that antibiotics don’t kill viruses) is different from climate science knowledge. Kahan set out to design a set of test questions that would actually dig in to the knowledge people possess, rather than how they felt about the public debate.

There's precedent for this work. Others have successfully added "According to the theory of evolution..." before questions to discover what people knew rather than what they personally believed. So Kahan prefaced questions about various aspects of climate change with "Climate scientists believe..." The questions asked about things like whether melting Arctic sea ice would raise sea level, which atmospheric gas is responsible for anthropogenic warming, as well as whether global warming would lead to increased risk of skin cancer. The goal was to provide as many tempting but wrong options for people on either side of the ideological divide.

Whereas questions about whether one believes anthropogenic climate change to be a real threat produced highly polarized responses, there was no significant difference on the rephrased questions. The average person who believes that humans are responsible for climate change answered half of the questions correctly—and so did the average person who believes humans have had no effect, or that the globe hasn't even warmed.

There was a strong correlation, though, between scores on the general science literacy test and this climate science literacy test. That's a good sign that it was a fair test of knowledge.

There was also an interesting tilt in the responses. People were generally much less likely to recognize false statements about things that are not risks—they tend to think the link between climate change and skin cancer is real. That suggests that many people were responding based on general ideas about climate scientists thinking climate change is bad rather than actual knowledge about each detail.

In addition, asking what scientists "believe" is different from asking what the evidence indicates—the latter would demonstrate real familiarity with the science. That's a bar that few of us can meet on multiple subjects, though, and it isn't how most people will get their information about climate change.

These issues, however, don't change the fact that those who scored highest also scored highest on the general science literacy test. As you would predict, they must have known enough to be better at dodging this tilt trap.

What's more difficult to understand is the difference between these results and studies that ask people whether they think there is a scientific consensus about climate change. Those who believe humans are not responsible for global warming greatly underestimated the degree of scientific agreement that humans are, in fact, responsible. However, these same people were apparently perfectly willing to agree that "climate scientists believe" there will be coastal flooding as a result of anthropogenic warming, for example, which seems to imply a scientific consensus.

It’s unclear what to make of that, Kahan told Ars, but it gets him wondering about the responses people have given in past studies. It could be that there’s some degree of compartmentalization going on, with people answering differently depending on whether they’re thinking about the public controversy or the textbook details. Perhaps saying that scientists are divided on the issue is another way of saying you don’t believe it.

On the other hand—and Kahan thinks this is more likely—these could simply be coming across as completely different questions in people's minds. For some reason, the questions in this latest study successfully ask people what they’ve heard about the scientific consensus, whereas the previous questions failed to do so.

Still, this study offers a reminder that many people have heard (or can guess, at least) a lot about what the science has to say. Kahan says this survey was far from a perfect “climate science literacy” test, but it shows that such a test is, in fact, possible—if you ask the questions in the right way. Some of us might like to see people know more about the science, but Kahan’s work argues that a lack of knowledge isn’t what makes climate change contentious.

A recent campaign has focused on communicating the strong consensus among climate scientists, a pithy distillation meant to help bring an end to the public debate. Kahan thinks that his work and others like it show that people pretty much know about that already and that repetition won’t make any difference. The problem is that a culture war has infected the conversation.

Advances in Political Psychology, 2015. (In press)