Moral Values Important in the 2004 Exit Polls

In a New York Times op-ed piece shortly after the election, ABC's Director of Polling, Gary Langer, argued that a "poorly devised exit poll question and a dose of spin are threatening to undermine our understanding of the 2004 election." Langer decried the phrasing of an exit poll question that asked respondents to select "the ONE issue that mattered most in deciding how you voted for president."  The results:

Langer objected to including moral values along with the other six items, because, he wrote, "While morals and values are critical in informing political judgments, they represent personal characteristics far more than a discrete political issue. Conflating the two distorts the story of Tuesday's election." Langer went on to note that six of the items on the list were "concrete, specific issues. The seventh, moral values, is not, and its presence on the list produced a misleading result."

But Howard Schuman, emeritus professor of sociology at the University of Michigan and one of the foremost researchers in the world on question wording, disagrees. Taking direct issue with Langer's argument in the online magazine, Public Opinion Pros, Schuman writes that whether "moral values" constitutes a concrete or specific issue parallel to the other issues, "the goal in framing a survey question is ordinarily to capture words that are meaningful to people, whether or not all the words are exactly at the same level of abstraction. To the extent that 'moral values' is meaningful to a substantial number of Americans in deciding how to vote, it is a legitimate alternative."

Was the phrase "moral values" meaningful to a substantial number of Americans? The exit poll suggests it was. As shown in the table, among the 22% of voters selecting the item, 80% voted for George Bush, just 18% for John Kerry. Among people who selected the next most popular item, "economy/jobs," the vote was reversed: 80% for Kerry, 18% for Bush. Clearly, each item helped to differentiate Bush voters from Kerry voters.

The other items, all considered legitimate by Langer, also showed large differentiations in the vote, with voters who selected Iraq, healthcare, and education choosing Kerry by a roughly 3-to-1 margin, and voters selecting terrorism choosing Bush by a 6-to-1 margin. Taxes showed the least differentiation: Voters selecting this issue favored Bush by a relatively modest margin of 57% to 43%. Still, all seven items provide insight into the factors that were related to voters' preferences.

One of Langer's other arguments is that pre-election polls using an open-ended format found few respondents, less than 10%, volunteering "moral values" as an important election issue, but in the closed-choice format of the exit polls, 22% selected the item. The higher percentage on the exit polls is thus an artifact of the polling method, rather than a genuine reflection of what voters were thinking. And rather than being one of the most important factors in the public's vote, "moral values" was at best a minor factor.

Schuman takes issue with this argument as well. He dismisses the notion that an open-ended question format is the "gold standard" for judging the importance of issues. In a 1987 Science magazine article, he and his colleague, Jacqueline Scott, reported on an experiment in which an open-ended question asked respondents to name the most important events of the past half-century. "Invention of the computer" was rarely mentioned. But in a subsequent poll, when the most frequently mentioned responses in the open-ended format were included along with the computer, the computer won out.

One of the problems with the open-ended format, according to Schuman, is that it is especially subject to "salience effects." As Schuman notes, "Forced to come up with a response on their own, people are apt to be influenced by what they have heard most recently on television or other news sources, and Iraq and terrorism have been much more in the news than the summary phrase 'moral values.'" Schuman's conclusion: "A closed set of choices can offer a more even playing field, provided it does, indeed, include all the main issues that respondents may wish to consider."

Since the election, articles in numerous magazines and newspapers have focused on the role of values in American politics. Given the results of the exit poll, despite the linguistic impurity of its issues question, that focus seems justified.