Behind the Controversy By Mark Lindeman and Rick Brady Part II page2

By their nature, national exit polls impose enormous personnel challenges. In telephone polls, it is feasible to hire and train relatively small staffs, who work in call centers under any desired level of supervision. To cover almost 1,500 precincts all around the United States for an event that occurs only once every two years, compromises are inevitable.

According to Edison/Mitofsky's postelection evaluation, about 23 percent of the interviewers for the 2004 election had previously worked as exit poll interviewers. After the election, 14 percent reported that they had been hired within two weeks of election day—7 percent “within a few days.” The recruits went through a “multi-step training process,” but only one step—a twenty-minute “Training/Rehearsal Call” that “included a detailed question and answer dialogue on all facets of the job”—is likely to have assessed to any extent whether the interviewers understood their instructions.

Moreover, about one-third of interviewers were twenty-four years old or less, and they attained substantially lower completion rates than older interviewers (50 percent, compared to 61 percent for interviewers ages fifty-five and over). As Edison/Mitofsky note, this result does not necessarily indicate that the younger interviewers did anything wrong; but, for whatever reason, voters on average were less likely to cooperate with them.

Exit poll codebooks sometimes blithely assert that respondents are chosen by random selection, and that the interviewer has no control over respondent selection. In pursuit of this desired objective, each interviewer is assigned an interviewing rate, which in 2004 ranged from one (approach every voter) to ten (approach every tenth voter). The interviewing rate is based on expected turnout at each precinct, chosen so that approximately one hundred interviews are conducted; the rate can be adjusted during the day if necessary. Interviewers are instructed to adhere strictly to this protocol, except during breaks. In principle, if interviewers arrive when the polls open and conduct interviews until they close, each voter within a sample precinct should have an equal probability of being selected.

Simple as this randomization protocol sounds, actual events can undermine it. First of all, interviewers simply may not show up when the polls open (often at 6:00 a.m.), and they may not remain until the polls close. Indeed, they are instructed to telephone in their final results in the last hour before the polls are scheduled to close, which means they are likely to underestimate late voters.

Worse, just five days before the election, Ohio secretary of state Ken Blackwell ruled that exit poll interviews should be treated as "electioneering," and therefore kept at least one hundred feet away from polling places. A judge overturned the ruling at 10:30 p.m. on the day before the election, but many interviewers were delayed in beginning their interviews until local election officials had been apprised of the outcome—in one case, close to 5 p.m. (We do not know how many interviewers delayed the start of interviews, but Edison/Mitofsky report that their legal team “dealt with legal and distance issues” in fourteen of the fifty Ohio precincts.) Since in many places later voters tend to be more Democratic, delayed access to the polls could have contributed to a Kerry bias in Ohio, whereas ending interviews early would have tended to induce a bias toward Bush.

Second, even when interviewers were in place, that “place” may have been far from the voters. Some interviewers were forbidden to stand near the polls because of state law, restrictive interpretations by state and local officials, or the discretion of individual precinct officials. As the required distance increased, voters had more opportunities (inadvertently or deliberately) to evade interviewer approach.

Moreover, where interviewers were held at the same distance as “electioneers,” voters might have been more likely to conflate them with electioneers. For instance, according to “Mystery Pollster” Mark Blumenthal, a Minnesota college professor who recruited four students to participate in the exit poll there reported that “at least one of my students was hampered by the fact that a contingent of folks from MoveOn.Org was stationed right next to her at the 100-foot line.” In the roughly three-quarters of precincts where interviewers were allowed inside the building or within twenty-five feet, the “miss rate,” based on interviewer records of how many prospective respondents they could not approach, was about 10 percent. Miss rates—and disparities between the exit polls and the vote count—increased rapidly as the distance increased. Unrecorded misses may well have been even higher.

Third, at least some interviewers exercised some discretion in who they interviewed, despite explicit instructions not to do so. The Minnesota professor reported after the fact that “nearly all” the students mentioned people being included not at random. Most often this happened when two people came out together and the person initially approached refused to participate, but the partner volunteered. “None of the students saw any ‘difference’ in which one of the two participated as long as one of them did.”

It seems likely that other interviewers took the same liberty. One student interviewer reported that a voter had taken a survey, filled it out, and placed it in the box without being approached, while the student was speaking with other voters. In 2004, as in some other exit polls, error rates increased as interviewing rates did—presumably, in part, because interviewers were more likely to deviate from the protocol (and also because larger, busier polling places provide more opportunities for voters and interviewers to miss each other).