Consider as prime examples of the problem the following presidential approval questions asked in one form or another by virtually every national and regional poll in America: "Do you approve or disapprove of the way George W. Bush is handling his job as president?" (Gallup). "How would you rate the overall job President George W. Bush is doing as president: excellent, pretty good, only fair, or poor?" (Harris). Amorphous in meaning and highly ambiguous in what they refer to, each of these questions will almost certainly mean different things to different respondents. What does "handling his job as president" mean? For some respondents it may mean mostly how he's handling some current international crisis; for others it may refer to how he's dealing with various economic matters; and for still others, how he's providing moral leadership for the country; and so forth. For that matter, what does it mean to say you "approve" or "disapprove" of the way he's handling his job? Similarly, what is the meaning of such vague quantifiers in the Harris question on presidential approval as "excellent," "pretty good," "only fair," and "poor"? Given the implicit presumption that they have an opinion (the opinion presumption), most respondents will answer these questions by making the interpretability presumption: How he's handling his job as president means what it is obvious to me here now that it means. And that will likely be different things for different respondents, which is just another way of saying they are answering different questions. Theoretically speaking, it is if the wording of the question itself had been changed for different respondents. The comparability issue is identical.

The same goes for the other equally vague measures of American public opinion that have become the stock in trade of so many polls today. "Do you think things in this country are generally going in the right direction or do you feel things have gotten pretty seriously off on the wrong track?" (ABC/Washington Post). "How much of the time do you think you can trust the government in Washington to do what is right: just about always, most of the time, or only some of the time?" (Gallup). "How would you rate the economic conditions in this country today—as very good, somewhat good, somewhat poor, or very poor?" (Gallup). "Do you think it will be best for the future of the country if we take an active part in world affairs, or if we stay out of world affairs?" (Gallup). "In your opinion which of the following will be the biggest threat to the country in the future—big business, big labor, or big government?" (Gallup). As Stanley Payne complained, unsuccessfully, about survey questions years ago, abstract concept words such as "right direction," "wrong track," "the government in Washington," "economic conditions in this country," "future of the country," "world affairs," "big business," "big labor," and "big government" are loaded with ambiguities. As a result, respondents may give different answers to questions using such terms, not because of genuine differences of opinion on the subject, but rather because they interpret the question differently. The typically sizable demographic differences between whites and African Americans, for example, in their approval rating of President George W. Bush may reflect in large part a fundamental difference in how they interpret the meaning of "handling his job as president." So it goes too for the "gender gap," the "marriage gap," the "generation gap," and other sociodemographic gaps in public opinion. Some, perhaps much, of what appears to be a significant difference of opinion between various demographic categories of respondents may reflect simply differences in how they are interpreting the meaning of questions about abortion; affirmative action; government spending on health care, welfare, foreign aid, and national defense; Social Security; Medicare; prescription drugs; cloning; and the like—a hypothesis that is readily testable. Such gaps, in other words, may be no more than an illusion, an artifact of measurement due to differences in the interpretation of survey questions that vary in ambiguity.

Excerpted from The Illusion of Public Opinion: Fact and Artifact in American Public Opinion Polls, by George F. Bishop. Copyright © 2005 by Rowman & Littlefield Publishers, Inc. Reprinted by permission. George Bishop applies his analysis of the illusion of public opinion to the exit polls from the 2004 elections. See his op-ed article in this issue of Public Opinion Pros.

Readers who wish to respond to this or other articles appearing in Public Opinion Pros, or to contribute commentary of their own in 800 words or less, should consult our author submission guidelines and editorial policies under "Letters to the editor and op-ed articles."