Standing Up and On the Way Out by Howard Schuman

On election day in 2004, two of the four versions of the major national exit poll began by asking voters to place checkmarks by their gender and their race or ethnicity, by the candidate they had voted for, and by the timing of their vote decision. Next, they were to respond with a checkmark to what seemed a crucial question: "Which one issue [out of seven listed] mattered most in deciding how you voted for president?" Several questions later they were presented a parallel checklist question about which of seven "candidate qualities" mattered most in their decision.

The "issues" question became a focus for interpretations of George W. Bush's victory immediately after the election, especially the modal choice of "moral values" as a major issue. My concern here, however, is not with the arguments over that choice, but with the nature of the issues question itself, and with the candidate-quality question as well. We need to consider the physical setting and self-administered nature of the exit poll questionnaire, and how these shape its format, which in turn in 2004 created the acrimonious controversy over the meaning of responses about issues.

A distinction between "issues" and "qualities" makes good sense from an analytic standpoint, and it might well be an important part of any study of why people vote the way they do. But it does not necessarily fit the way voters think about their presidential or other electoral choices. Indeed, as Philip Converse observed several decades ago, there is substantial evidence that many Americans do not make a sharp distinction between issues and personal qualities of candidates when deciding how to vote, but think largely in terms of the one or the other or mix the two together in ways difficult to disentangle. For example, in 2004, one of the seven choices to the issues question was the word "terrorism," but quite likely this was answered not in terms of supposedly different positions by the candidates toward terrorism, but in terms of which candidate had the necessary personal qualities to prevent or respond to a terrorist attack.

Moreover, some voters may have explicitly based their vote on neither issues nor qualities, but on the political party of the candidate, his religious or ethnic identity, or some other factor quite different from any that might appear on either an issues or a candidate-quality checklist. At the same time, the list itself might suggest reasons for one's vote that were not entertained ahead of time. We cannot know the extent to which that happened with the 2004 exit poll, but there were claims that such response bias may have inflated the percentage choosing "moral values."

It would seem wise to begin a survey of voters' explanations for their votes by asking an open question that does not impose any structure on responses. Some preelection polls did just that. For example, a Pew Survey carried out in October followed candidate preference questions by asking in open-ended form: "What is the main reason why you would like to see [Bush/Kerry] win the presidential election?" The results indicate that many respondents answered in terms of either issues or qualities, but not both, and that with qualities there was a frequent division between stressing positive qualities of one's chosen candidate and stressing negative qualities of his opponent. Converse's 1964 analysis provides the classic treatment of open responses for addressing the way in which voters think about their decision.

Of course, coding open responses is bound to be messy, as anyone who has done such coding will know. Both definitions of codes and coding itself have an ineluctably subjective element. Yet the actual responses are available to remind analysts that the borders of conceptual codes are imprecise at best, whereas with closed questions it is all too easy to proceed as though the alternatives offered carry the same meaning to respondents as they do to the investigator.

Why, then, did the exit poll not start with such an open question, especially since it could then have been followed by more specific closed questions of the kind already in the exit poll questionnaire?

The explanation is almost self-evident. The exit poll interviewer must identify and capture randomly designated voters leaving the polling place, and, as the Mitofsky-Lenski report on the exit poll makes clear, this was difficult and frequently unsuccessful. Turnout was high, the polling locations were often crowded, and when people had already waited in line to vote and were in a hurry to leave, they could not be presented with more than a small number of closed questions that called for quick checkmarks.

In many locations voters would not have had a comfortable place to sit in order to write out an open response, even if they were willing to do so. Nor was the "interviewer" really an interviewer, but rather a kind of proctor, with the overriding task of identifying and persuading the next nth person to stop and fill out a questionnaire. Thus, the questions were self-administered, and there would not have been time even to determine if an open response was legible, let alone to probe a confusing answer.

The physical and temporal setting of the exit poll determines the format of the questionnaire, which in turn shapes the data available to journalists and others eager to receive the first results. With these constraints clearly in mind, careful pretesting is needed to decide on those few easily answered closed questions that will be least ambiguous and most informative in the early hours after the election, the more so since their initial use will be mainly in the form of single variable distributions, plus simple differences between the supporters of each candidate.

Possibly, systematic pilot studies, starting with open-ended inquiries, could lead to a basic closed question that would list all the main explanations for voting, including issues, qualities, and other factors like party identification. Such a list is likely to be both very long and quite heterogeneous in content, and whether it would be practical as part of a self-administered exit poll calls for serious investigation, perhaps with respondents giving their first, second, and third preferences in order. Such a list still risks suggesting issues, qualities, or other considerations as the voter rushes through it, but preliminary experimentation could determine whether this happens to a prohibitive degree.

None of this is to say that the data from an exit poll are not extremely valuable for understanding how voters explain their own decisions about their vote. But it is essential to keep the limitations of the exit poll format in mind when interpreting the questionnaire beyond its use to project the results of the election itself, which of course has its own problems. The questionnaire should be thought of as a set of rapid inquiries that are usually answered while standing up and on the way out.

Howard Schuman is a professor and research scientist emeritus, University of Michigan. Most of his research has been done through the university's Survey Research Center. He is the senior author of Questions & Answers in Attitude Surveys: Experiments on Question Form, Wording, and Context (1981;1996) and of many other articles on survey questions.