Weighting It Out: Party Identification and Election 2004 By Larry Hugick and Stacy DiAngelo

The Newsweek poll and Princeton Survey Research were by no means the biggest targets for attacks over party ID in election year 2004. The Gallup Organization took the brunt of them. Certainly one important reason for this is Gallup's high visibility as the best-known name in the polling industry and their high frequency of polling for their clients, CNN and USA Today. In my opinion, however, Gallup's use of a likely-voter methodology developed for use in the final week of the campaign throughout the entire year was also an important factor. Reporting likely voters early on often meant that their results were based on fewer interviews and a larger margin of error than those of their competitors. And questions that work very well in identifying actual voters very close to election day are less accurate farther out, with the potential, unintended consequence of producing a partisan mix that overrepresents one party or the other.

The Gallup likely-voter methodology was the subject of the famous full page attack ad by MoveOn.org that appeared in the New York Times. This attack ad demonstrated the level to which partisan attacks on nonpartisan pollsters had sunk. Those leveling the charges of bad polling methods began to do more than simply cite statistics; they speculated about motivation. The MoveOn.org ad implied that there was a connection between Gallup chairman George Gallup Jr.'s evangelical beliefs and his organization's polling, which had often been more favorable for Bush. This is an absurd charge to anyone who knows George Gallup or the people who run the CNN/USA Today/Gallup poll. Other media pollsters were assigned less conspiratorial motives, like simply trying to pump up a story.

Fortunately, there were some voices of reason that cut through the ill-informed clamor about flawed poll methodologies. A news release by Andy Kohut and the Pew Research Center in mid-September provided the helpful antidote--actual data--to counter the proposition that party identification is a fixed characteristic. Cliff Zukin, who became president of the American Association for Public Opinion Research this past May, took the initiative to put together an advisory to poll-watchers with a cautionary note that most pollsters do not regard party ID as a fixed characteristic or use it as a weighting variable. Sharon Begley, a science writer for the Wall Street Journal, wrote an article about polling methods that did not simply repeat the "some weight by party ID, some don't" theme, but boldly stated that "every scientist I asked has grave qualms [about party ID weighting]." Mark Blumenthal, a Democratic pollster with academic credentials in survey research, started the excellent Mystery Pollster website and posted a series of articles about the party ID weighting issue, providing needed perspective and informed commentary accessible to both insiders and those outside the polling community.

In the end, the proponents of party ID weighting were essentially proved wrong by the election outcome and the party breakdown results from the national exit poll, which showed equal numbers of Democrats and Republicans instead of the four-point Democratic edge recorded in 2000. Before closing the books on the party ID issue, however, it is important to look back on the experience of 2004 and what can be done to better educate people about the issue so this experience is not repeated in the future.

Next month in Public Opinion Pros: Trends in party identification, 2001 to 2004.