Four Calls or Less: A Dispositional Model of Preelection Polls

By David Dutwin and Melissa Herrmann

An important part of each election cycle is what might be called the “assessment phase,” when researchers cope with the enormous amount of data accrued during the most recent election to determine how better to refine their craft and enhance polling accuracy for the next cycle.

Included within the literature produced by these assessments is a vast documentation on likely voter models, as well as articles concerning overall polling accuracy. A different, though less common, take on election polling is to investigate potential bias by the actual “trade” of survey research.

Yet from a survey research perspective, no question should be more important than whether the methods and procedures used in preelection polls are sound. Preelection polls have “institutional constraints” that make them more similar in quality to market research surveys than to social scientific surveys:

• Response rate is typically not an issue and, in many cases, is not even measured. People who refuse to respond to telephone surveys are rarely, and then only lightly, dialed back for conversion. Also, many survey firms use unidentified phone extensions, such that numbers dialed from a CATI system will be blocked by call zappers and privacy managers. With high response-rate projects, such numbers would be called back on an openly identified line.

• The field period of most preelection polls is five days or less. This denies researchers the possibility of extensive callbacks, although in rolling cross-sectional situations (tracking polls) most researchers will roll over callbacks into following waves or days.

• Most political polls do not invest the time or money to interview in Spanish. And, as might be expected, there is a clear difference between Hispanics and non-Hispanics on a range of political attributes.

• Finally, the call attempts in any short-field-period study are necessarily limited, typically to four or fewer.

We used as the basis for our analysis Health Care Agenda for the New Congress, a November 2004 national adult survey by the Kaiser Family Foundation and the Harvard School of Public Health—obviously, not a preelection poll. Our strategy was to utilize a poll that contained self-reported voting behavior and was also more methodologically rigorous than the typical preelection poll in order to make comparisons among differing rates of contact within each disposition type. The study employed a number of traditional demographic measures and sample variables, as well as variables measuring party identification, voter registration, voting behavior, and ideology. Finally, the study measured the number of call attempts required to attain a completed interview, as well as the “pathway to completion” dispositions for each completed interview.

The primary research question for our analyses was, are there systematic variations by disposition with regard to the most critical variables in preelection polling—vote behavior and its most powerful corollaries, party identification and voting/registration behavior? A secondary question was whether there was systematic variation by disposition with regard to demographic and sample variables.

Because our main goal was to explore the methodological validity of typical preelection poll methodologies, we broke the main research question into two subtopics. The first was the question whether it is justifiable to expect nonbiased answers in polls that attempt, typically, four call attempts or less. As such, one of our regression analyses was designed to explore the differences between completes that occurred from call attempts one through four, compared to five or more attempts. The second subtopic, of course, regarded the question of dispositions as described above. For these analyses, cases were limited only to completes that were attained at one through four call attempts, in order to mimic the types of completes that would be attained in a typical preelection poll.