Best Foot Forward: Social Desirability in Telephone vs. Online Surveys by Humphrey Talyor, David Krane and Randall K. Thomas

The results for the ratings of the items obtained from the assigned online group are shown in Figure 1. Six items were clearly rated more desirable (with means above 3 on a scale of 1 to 5) and five items that were rated as undesirable (with means under 3).

Figure 2 summarizes the results we obtained for the self-reported behavior for the items. They strongly suggested a much lower level of "socially desirable responding" in the online survey, consistent with our original hypothesis.

Looking across the items, we can see that the size of the difference in response patterns between the online and telephone surveys varied greatly, and generally in the predicted direction. However, the difference in responses was not necessarily linked directly to the rated level of social desirability of the item shown in Figure 1. For example, the rated goodness of doing volunteer work was the highest of the items (mean = 4.49), but there was little difference between the two interview methods (49 percent for online versus 52 percent for telephone); while the rated goodness of going to church, mosque, or synagogue was lower (mean = 4.12) but showed one of the greatest differences (25 percent for online versus 56 percent for telephone). Therefore, while an item's rating along the scale of "good" or "bad" distinguished the direction of the difference between interview methods, as shown in Figures 1 and 2, it was not particularly informative concerning the size of the difference. Most likely, there are factors other than rated "goodness" that affect the size of the difference between the two methods, such as the extent of embarrassment a person feels when admitting to less desirable actions or thoughts, or the extent to which a less-desirable behavior or thought can be attributed to the character of a person rather than the situation the person is in. If a measure of such other factors could be developed, we might be able to use it to predict more accurately differences in item response between the two interview methods.

The biggest differences suggest that Americans are particularly embarrassed and uncomfortable when talking about their religious observance (or lack of it), their lack of exercise, their atheism or agnosticism, and their dental hygiene. One suspects that other countries might have very different levels of social desirability, and that different variables would be higher or lower on their lists. But as observers of the American scene, we are not at all surprised that many Americans appear to be reluctant in telephone interviews to admit to not believing in God, to not attending religious services, to drinking alcohol, and to not exercising regularly. We are also not surprised by other substantial, if smaller, differences, with many Americans apparently unwilling to tell interviewers that they often drive over the speed limit, do not give money to charity regularly, gamble, or have been diagnosed with depression.