Weighting It Out By Larry Hugick and Stacy DiAngelo part3 page4

The frequency with which trends in party ID change direction raises serious questions about another way some pollsters manipulate their data—weighting survey data to party ID distributions from recent surveys of the past. Instead of actually improving survey accuracy, this practice would seem to distort the results by filtering them through the political environment of the past.

While party ID can and does change quite often, it moves in a relatively tight range. In individual Newsweek polls from 2001 to 2004, the percentage Republican ranged between 26 and 35 percent, while the percentage Democratic ranged between 26 and 37 percent. This range is larger than might be attributed to sampling error alone, but not by very much. Based on sampling error alone, we might expect estimates for each party to vary by about seven percentage points (three and one-half points in either direction). Because of sampling error, differences in party ID distributions from poll to poll are rarely large enough to be statistically meaningful.

Those who believe party ID is a fixed characteristic are quick to criticize individual polls for having “too many Republicans” or “too many Democrats.” Aside from the issue of real movement in party ID, random sampling error by itself can produce results that seem aberrant to the untrained eye. Pressure from such critics may lead some pollsters to give serious consideration to using some form of party ID weighting to “smooth out” their results.

The problem with taking this path is that, at any given time, one cannot distinguish real change in party ID from statistical noise. If party ID is to become a routine part of reporting by pollsters during an election year, displaying recent highs and lows in the percentage Republican and percentage Democratic along with the current figures may be one way to provide perspective and help people better process this information.

Probing demographic patterns in major party ID shifts offers further insights about the dynamics of such change—and further evidence that party ID movement reflects real-world change, and is not just random noise. This study identified two major increases in Republican party identification in recent years, the first in the months following the terrorist attacks of September 11, 2001, and the second after the GOP convention in September 2004. The movement toward the GOP after 9/11 came primarily from subgroups less likely to lean Republican prior to the terrorist attacks. This includes Hispanics, blacks, and whites who live outside the South. The most Republican subgroup in the population—southern whites—did not change significantly in party ID after 9/11.

In contrast, the post-convention GOP surge that foretold George W. Bush’s reelection had very different demographic patterns. In this case, southern whites were one of the groups that most dramatically increased their numbers of self-identified Republicans.

These demographic patterns strongly suggest that party ID movement is linked to the news environment at the time the polling is done. The post-9/11 surge followed the pattern of a classic “rally-'round-the-flag” effect, in which those less inclined to support a president (and his party) rally around him during a time of war. The post-GOP convention surge showed a different pattern, in which a presidential candidate rallies his political base, in addition to attracting support across the political fence.

The media polls received the most criticism about their “flawed” party ID distributions after the September 2004 GOP convention. Although the idea of a post-convention “bounce” for the incumbent in the presidential horserace was widely accepted by the political community, many were skeptical about the apparent “bounce” for the Republicans in terms of party ID that was apparent in a number of major polls.

What was missing at the time was an historical perspective—what did polling in past election years tell us about changes in party ID around conventions? Going back as far as the 1984 election using Newsweek poll data, we can conclude that a “bounce” in the polls for an incumbent party after its presidential nominating convention is not at all unusual. In five of the last six presidential election years, Newsweek poll data found the party-strength margin shifting by six to eleven points in the direction of the incumbent’s party. Only in 1992, when George H.W. Bush was on his way to being defeated in his bid for reelection did the incumbent party’s candidate fail to get a boost. Assuming that the demand for party ID results continues in future elections, it is important to get this information about historical patterns to poll-watchers so they are better able to put post-convention poll results in perspective.

The relationship between the horserace and party ID is an obvious one, and is the main reason why many poll critics carefully scrutinize the party ID distribution and use it as a basis for determining accuracy. They view party ID as an anchor that people use to organize their political thinking. In other words, people first decide if they are Republicans, Democrats, or Independents, and that, in turn, is a major factor influencing their choice of a presidential candidate. There is little question that party ID and presidential preference are strongly correlated. Their trend lines move together during an election year.

However, this theoretical model of party ID as an anchor is not well-supported by empirical evidence. The findings of our analysis of the Pew Research Center survey data from interviews with the same voters at two points in the fall campaign make a strong point that, at the very least, party ID is no more of an anchor for organizing political thinking than the horserace. While 14 percent of respondents in the Pew Research Center callback survey changed their presidential preference between September and October 2004, 18 percent changed their party ID. Close to one-third of those who changed their presidential preference also changed their party ID.

The controversy surrounding party ID that characterized election 2004 may not be repeated in future elections. Whether it is or not, the demand for party ID information is likely to continue. Our job as pollsters is to find ways to educate poll consumers and give them more than just the figures. The internet has made it easy for noninsiders to access the kind of information about polls that would have never been available to them in the past. While disclosure of party ID information in such an environment seems appropriate, it needs to be accompanied by a greater effort to educate people about, as Andy Kohut put it, “what it is and what it isn’t.”