Contradictory Poll Results? What a Great Opportunity!

The premise under which I operate (and the same for our work here at Gallup more generally) is that public opinion is massively important. And that it should be paid close attention to.

But not everyone agrees.

For example, there is the "confused" argument. Politico writer Andie Coller penned a piece on Monday reusing the well-worn premise that public opinion is contradictory and confused, and therefore difficult to understand and make use of. In this instance, Coller is using the example of research on health care reform. Her title "Health Care Polls Leave Pols Dizzy" conveys her premise (although this conclusion as represented in the title is conjectural; other than a reference to Nancy Pelosi not liking the wording of a question on the public option in one poll, there are no actual politicians quoted in the piece who, in fact, say that they are left dizzy by health care polls.)

Coller's piece builds off the not-shocking finding that the results of the wide variety of recent polls on health care reform can often be difficult to make sense of. She calls these results a "...dizzying deluge of hard-to-reconcile data". She then goes on to list examples wherein health care polls seem to produce contradictory results.

This is not a new observation. In fact, I devoted a chapter of my book Polling Matters to the issue of interpreting what appear to be conflicting or confusing estimates of public opinion on key issues.

The key is to flip the premise. It isn't that the public is "confused," but that researchers have not taken the time to examine carefully the context and meaning of responses to questions that ask about different aspects of complex issues.

The idea is to learn from the data rather than to dismiss the data (much like an evidence-based analysis which tries to make sense out of differing results in tests of medical procedures and drugs.) If people react differently to differently worded questions about aspects of health care reform, as they often do, then the question is "why?" What is the wording or premise of specific questions that are related to differential patterns of answers? Putting it all together advances understanding and advances the ball down the field. That's how science operates.

For example, a recent CBS News/New York Times poll found majority support for a public option explained as a "government-administered health insurance plan -- something like the Medicare coverage that people 65 and older get". A contemporaneous NBC News/Wall Street Jounal poll found less than majority support for a public option explained as "a public health care plan administered by the federal government..."

The important issue: why the difference?

Well, in part, we can hypothesize that Americans like the idea of a health insurance plan more than a health care plan. And in part, we can hypothesize that reminding Americans that the plan would be similar to the familiar Medicare system makes them more positive. These differences can be tested in controlled experiments. But drilling down to exactly what's behind the different reactions expands our knowledge about American public opinion on this crucial topic. We certainly don't want to throw both poll results out the window because on first glance they appear contradictory. Differences in science lead to discoveries. Scientists don't dismiss anomalies but analyze them.

(And not all research on health care reform is contradictory. When one looks at poll questions asking if Americans favor or oppose a new health care reform law, Obama approval on health care, Obama versus Republicans on health care, personal satisfaction with current health care, one is struck by the consistency of the findings rather than any differences.)

Another way to handle public opinion on complex answers is to advance the "dismiss it" argument.

I recently heard from a thoughtful and highly engaged entrepreneur who had very strong views on health care reform. When advised (by me!) that the public had different views, he simply said that the public was wrong. And dismissed the public's views as ill-informed and poorly reasoned.
This type of elitist argument has been used against public opinion for many years. If the public disagrees with one's premise or one's conclusion, then one dismisses the public. In this particular instance, the entrepreneur was arguing that Americans have lousy health care, particularly in comparison to health care in many European nations. Informed that the significant majority of Americans say they are satisfied with their health care, the entrepreneur advanced the argument that the public in fact stupidly didn't realize that their health care wasn't that good. They just didn't know any better.


My argument is that there is little dispute of the social fact that Americans are satisfied with their health care. That has been well established. One can argue that Americans' consciousness should be raised to the point where they realize that there are ways to improve their health care along the models of select European countries. But one should not dismiss the basics of public opinion before doing so.

Both the "confused" and the "ill-informed" approach to public opinion are short-sighted. And deflect us from the wonderful opportunity to take advantage of the wisdom of average people when their opinions are pooled together.