From Norbert Schwarz

Professor of Psychology, Research Professor, Institute for Social Research, Professor of Marketing, Ross School of Business, University of Michigan

I’ve been thinking about this interesting question for the last couple of days: What have we learned since Questions & Answers? Which issues have been “solved,” and which new ones have been added? Unfortunately, I’ve read too much Popper as a student to think we’ll ever have final answers on anything, but that’s a different story.

First of all, the question once again reminded me how well Schuman and Presser (1981) prepared the field for the cognitive turn it took a few years later. The book presented us with a clearly defined set of phenomena—some puzzling and some rather “obvious” from a social cognition perspective (which was a new baby at the time, with Wyer and Carlston’s 1979 book, Social Cognition, Inference, and Attribution, as the first “bible”). Without Questions & Answers, Hippler, Strack, and I—and probably many others—would not have been drawn to the interface of psychology and survey research. The book was the well-defined problem statement that gets an area going.

Second, thinking about this also made salient my disappointment that we haven’t really discovered “new” survey phenomena in the attitude domain that were not part of the book. There’s some new stuff on behavioral reports that came out of CASM, but as far as attitudes go, Schuman and Presser (1981) pretty much identified all the big issues for survey methodology, and later work mostly provided accounts of the underlying psychological processes.

So what do we know about how people answer attitude questions? Below I cite overviews, most of which are at my website. Here’s my (idiosyncratic) take:

• General Perspective. People form attitude judgments on the spot, mostly by constructing mental representations of the target and of a standard on the basis of information that comes to mind at that time. Which information that is depends on (1) how they construe the meaning of the question (a Gricean conversational story), (2) what was brought to mind by preceding questions (mostly a cognitive accessibility story), and (3) the order in which they consider the response alternatives (which prompt thoughts about the issue). None of this is an “artifact,” and all of it happens pretty much in the same way in the wild. The influence of natural contexts parallels the influence of question order; interviewing people at work, for example, produces similar answers as preceding the target question with work-related questions—in both cases, it’s work that’s on their minds, and this influences answers to all questions to which it may be relevant. The problem for surveys is merely that we mistake context-dependent answers for general ones—we learned how they think about X “when at work,” but treat it as if it were how they “always” think about X.

I’ve recently been arguing that, from the actor’s perspective, this “context dependency” is better framed as “context sensitivity,” and that it is highly beneficial. Any evaluative system that’s supposed to guide behavior needs to be context-sensitive (for the actor), but the observer doesn’t like that because it undermines prediction of the actor’s behavior. In my eyes, we’ve been doing attitude research from the observer’s perspective, and it’s time to try the actor’s perspective. That actor-observer distinction is actually something Schuman (1983) hinted at in Personality and Social Psychology Bulletin. I discuss this in a piece on “Attitude Construction” (written for a special issue of Social Cognition, forthcoming).

• Question order. I think we have gotten a pretty good handle on most question order effects. Some question order effects are primarily comprehension effects, and the meaning of the question shifts as a function of the context. That’s the Gricean story of my 1996 book, Cognition and Communication, and it also applies to graphical layouts, scale formats, and similar issues.

Other question order effects are a function of substantively related material that is brought to mind by earlier questions. I think we understand the conditions of assimilation (carry-over) and contrast (backfire), the determinants of their size, and the conditions under which the effects are limited to one attitude object or generalize over several objects. That’s the Schwarz and Bless inclusion/exclusion model (in Martin and Tesser, The Construction of Social Judgments, 1992), which is very much in agreement with Tourangeau’s model as far as assimilation effects are concerned, but adds an account of contrast effects that’s missing in the latter. Tourangeau discusses these issues in his chapter in Sirken et al. (1999), and a recent Schwarz and Bless piece is an update of our 1992 model.

Let me emphasize that Questions & Answers had an important impact here: Schuman and Presser's part-whole contrast effect identified a phenomenon that required the joint operation of cognitive and communicative processes, which is something that wasn’t known before. The story in Schwarz, Strack, and Mai's "Assimilation and Contrast Effects in Part-Whole Question Sequences" illustrates this interplay. So, in many ways, conversational and cognitive processes work hand in hand in question order effects.

We recently also learned that there is a nice cultural difference in the emergence of part-whole contrast effects. A short summary of that is in a 2003 piece on “Self-Reports across Cohorts and Cultures.”

Finally, preceding questions may bring norms to mind, as Questions & Answers illustrated with its norm-of-evenhandedness studies. I’m not aware of new work on that.

• Response order. This is a frustrating issue. In Thinking about Answers, we described a model that builds on Krosnick and Alwin and predicts an order x mode x item plausibility interaction. The model works fine if we write the response alternatives to be agreeable/plausible or disagreeable/implausible for most respondents. But most regular surveys present a mix of options that are differentially agreeable/plausible for different respondents, and we can’t make clear predictions for the sample as a whole. It’s an illustration for how a model that seems right theoretically, and works under tightly controlled conditions, is relatively useless for practitioners. As a psychologist it doesn't bother me (the mind isn't designed to make surveys easy), but as a survey methodologist it's very dissatisfying.

Making things worse, many survey tests of our model have dropped the “plausibility” variable. They essentially test what is theoretically a three-way interaction (order x mode x plausibility) as if it were a two-way (order x mode). The results are confusing—as they should be when you gloss over one crucial variable. George Bishop and Andrew Smith’s 2001 Public Opinion Quarterly piece illustrates this problem: The authors test a two-way interaction and conclude that a model that predicts a three-way interaction doesn’t work (see also Bishop's comments in this exchange). Unfortunately, such errors of analysis provide neither a basis for theoretical conclusions nor for applied advice.

The new thing we learned is that response order effects increase dramatically with age, in particular on the phone; see Knauper's 1999 study, "The Impact of Age and Education on Response Order Effects in Attitude Measurement."

• Nonattitudes/attitude strength. From an attitude construction perspective, the distinction between attitudes and nonattitudes is not theoretically meaningful. Moreover, there's no empirical relationship between attitude strength and susceptibility to context effects, despite popular beliefs to the opposite—Krosnick and Schuman (1988) put the nail into that coffin with a Journal of Personality and Social Psychology piece. But it leaves an interesting paradox: self-reported attitude strength does predict resistance to persuasion, but it does not predict susceptibility to context effects. I think I have an answer to why that is, and I discuss it in the “attitude construction” manuscript in a section on attitude strength (Schwarz, in press).

A special twist are questions about fictitious issues. In my reading, that was an artifact produced by researchers. Respondents have no reason to assume that we’d ask a meaningless question and peruse the context to make sense of it. Once they made sense of it, they can give a meaningful answer, as Schuman and Kalton (1985) noted. Our demonstration of that is Strack, Schwarz, and Wänke's Social Cognition, and I review it in chapter 4 of Cognition and Communication. The answers to questions about fictitious issues are not meaningless—only the questions are.

• Open versus closed. What we know is what Schuman and Presser (1981) said, and there are no qualifications to be added, at least none I’m aware of. I’m couching the differences in the language of Gricean conversational processes, where the response alternatives clarify what the researcher wants to know (chapter 5 of Cognition and Communication). This makes it look less “odd” (in fact, it’s perfectly consistent with proper conversational conduct) but doesn’t change anything about the robust effect.

• Don't know and middle alternative. Same thing. Nothing to be added to what Schuman and Presser (1981) said, I think. Except, perhaps, that “full filters” can convey the expectation that you should only answer when you are an expert. Hence, they discourage people from reporting opinions they would otherwise report (reviewed in Cognition and Communication, chapter 5).

• Tone of wording. Aside from Hippler and Schwarz (1986) on forbid/allow, we’ve never done much on that, nor have I seen much other work. Conceptually, I think that different tones convey differentially extreme meanings, which triggers a search for different information. For example, “What do you dislike about Bush?” versus “What do you hate about Bush?” would do so, and once that stuff has been brought to mind, you use it later on.

There are a few other topics in Questions & Answers that I have nothing to say on. Eric Knowles has done some interesting work on acquiescence, which is worth looking at.

That’s it. What really impresses me is how much Schuman and Presser (1981) set the agenda twenty-five years ago. What disappoints me is that while we did well in providing more process-oriented accounts of known phenomena, we didn’t put many new phenomena on the map (aside from extensions of familiar stuff, some age- and culture-related differences, some direction of comparison effects, and so on, but that’s all perfectly within the paradigm). So looking back, I feel that the effort was useful and certainly had considerable applied impact; but its pay-off in terms of novel psychological phenomena was more limited than I once hoped.

To Tom W. Smith

To first page