Presidential Approval Ratings in Perspective

By Larry Hugick, Jonathan Best, and Stacy DiAngelo

National polls have been asking the public to rate the president's job performance since 1935, during Franklin D. Roosevelt's first term in office. The standard version of this question—"Do you approve or disapprove of the way [president's name] is handling his job as president?" adopted by the Gallup poll in 1945, is now used in most, if not all, telephone polls sponsored by major national media organizations. Thirty years ago political junkies hungry for the latest results might have had to wait a week or two for a new round of presidential approval data; today hardly a day goes by without a new set of approval figures being served up by one organization or another.

While more data is generally better than less data, it is not without its pitfalls. Most seasoned poll watchers know that the best practice is to limit trend analysis to polls of one particular organization—rather than mix and match polls from different organizations—but there is a strong temptation to make cross-comparisons. Even if organizations use the same basic methodology and question wording, "house effects" caused by less obvious factors—such as the way a CATI system works and interviewers are instructed—can introduce differences to poll results.

Many political reporters, columnists, and other non-survey research specialists who frequently use approval data know less than they should about differences in methodology among the major national media polls they monitor and are misinformed about which differences are likely to have a significant impact on results and which are not. Issues such as variation in party ID distributions from poll to poll, the presumed political slant of the media organization sponsoring the poll, and purported day-of-the-week interviewing effects are brought up all too frequently by those who know less than they think they do about poll methodology. Those of us who conduct these polls for the media often don't spend enough time keeping track of what our competitors are doing, and our impressions of other pollsters' methodology may not always reflect current practices.

Are the approval ratings of these major media polls basically interchangeable, or do they differ in ways that can be quantified? To find out, we selected five polls, based on two criteria: The approval question posed had to make use of the standard Gallup wording, and it had to have been asked more than once a month, not including one-night polls.The polls that met these criteria were the ABC News/Washington Post poll; the CBS News poll; the CNN/USA Today/Gallup poll; the FOX News/Opinion Dynamics poll; and the Newsweek poll, conducted by Princeton Survey Research Associates International.

Of the five, the CNN/USA Today/Gallup poll was by far the most active, releasing forty-one separate sets of results over the twelve-month period, while the ABC News/Washington Post poll was least active, with sixteen approval measures in 2003. Two other prominent media polls that regularly ask the standard Gallup approval question—the CNN/Time poll and the NBC News/Wall Street Journal poll—fell below the activity frequency threshold. We were aware that the FOX News/Opinion Dynamics poll differed from the other four selected in their reporting base; registered voters rather than the entire voting-age population were sampled. However, our own recent polling suggested that the registered voter base—often representing over 80 percent of a general public adult sample—introduces only small differences to poll measures of political attitudes. What's more, we believed the study would be enhanced by including a poll sponsored by a top-rated cable network that has positioned itself as an alternative to the mainstream media, to see if its poll diverged in any specific way from the others.

The time period had to be as contemporary as possible to reflect current methods and the conditions that might affect our ability to conduct surveys, but at the same time provide sufficient data to analyze. In part, our decisions about which polls to study helped us make that decision, since the FOX News/Opinion Dynamics poll was not launched until 1996. Limiting the study to 1997-2003 (the period encompassing Bill Clinton's second term in office and the first three years of George W. Bush's presidency) seemed a reasonable compromise. It had the added advantage of variety, in that the two presidents being evaluated were from different political parties, with one in his second term in office and the other in his first.