Related Measures: Direction of the Country and Job Approval

By Larry Hugick and Stacy DiAngelo

The direction of the country question has become a staple of the national media polls. Every major polling organization asks a question to gauge people’s perceptions of how things are going in the country, or if they believe it is headed in the right direction. In nonpresidential election years, only the presidential job approval question is asked and reported more often. The direction of the country question sums up the public mood in a simple statistic, allowing us to compare Americans’ feelings about current conditions with how they felt during past times of national triumph and tribulation. Interpreting the results of these direction questions, however, is a challenge for professional pollsters, let alone more casual consumers of national media polls.

Unlike presidential approval, direction of the country seems to have as many different questions as there are polling organizations asking them. Most of the major media polls use some variation of the right direction/wrong track language, but even within this group there is no universally accepted wording. What’s more, our own organization (Princeton Survey Research Associates International) and Gallup, the most recognized name in the polling industry, use a completely different wording—asking people if they are “satisfied or dissatisfied with the way things are going” instead of whether the country is “heading in the right direction, or... on the wrong track.”

These differences give reason to doubt whether different organizations’ direction of the country poll results are directly comparable. Another reason is question order and placement. Nearly all polls that include the direction question also include presidential approval, a question that has been in use longer and typically appears on each and every poll. What is a pollster to do? Response to one question could conceivably influence response to the other. The more general nature of the direction question argues for putting it before approval, but the more frequent use of the approval question may convince others that it should come first. Using a third approach, some organizations, like Princeton Survey Research Associates International, split the difference and rotate the order in which these questions are asked.

For our study of the comparability of direction of the country questions, we classified the questions into three categories, to which we refer as the original wording, the modified wording, and the alternate wording, and then conducted an across-category analysis.

As the first step of our analysis, we removed from consideration those polls with only limited data and those that were problematic for other reasons. This reduced the list to four polls: one using the original “right direction/wrong track” wording (CBS News/New York Times), one using the modified “right direction/wrong track” wording (AP/Ipsos), and two using the alternate wording (Gallup and Pew Research Center). These four polls all asked the direction question frequently enough to permit us to calculate yearly averages for four consecutive years, 2002-5. By using the yearly averages to calculate a grand average for the entire period for each poll, we obtained the results shown in Table 1.

As the table shows, these questions got reasonably close results despite the variety of wordings used. CBS and PRC were almost identical. Gallup and AP were similarly very close to each other.

There are two key patterns worth noting here. First, the percentage of respondents saying “wrong track/dissatisfied” varied by only one point for data across all four organizations. Where the differences lay was in the “right direction/wrong track” and the “unsure” columns. With a sample size this large, the differences between Gallup and PRC and Gallup and CBS were statistically significant.

The differences in the “unsure” percentages noted here were not unique to the direction of the country question; the same pattern was found with regard to presidential approval ratings, and presumably other general questions. To allow for the best comparisons of these four polls, it was optimal to minimize the “don’t know” factor, and so we adjusted the grand averages for each poll in Table 1 so that the percentage “unsure” matched Gallup’s 3 percent. For each of the other three polls, the remaining “unsure” response was assigned to the positive response column. Table 2 shows these adjusted grand averages.

As the table indicates, the four organizations’ results were all within one point of each other in the percentage “right direction/satisfied.” Although we might have expected the modified wording to have a higher “wrong track” percentage because the wording did not specify “seriously” on the wrong track, there is no evidence to support this hypothesis. In addition, despite using different wording, the alternate Gallup version of the question did not get a different response than the original “right direction/wrong track” version of the question.