Methods and Sample Characteristics

Interviews for this study of advertisement effectiveness were conducted using a self-administered, online questionnaire, via HCD Research's proprietary, web-assisted interviewing software. The HCD interviewing system permits online data entry of interviews by the respondents. In addition, the software allows for the tracking of individual reactions to the survey.

Respondents for the political advertisement surveys were drawn from a nationally compiled database emanating from five hundred websites, providing opt-in or double-opt-in from individuals who were interested in participating in research and who had opted in for research within thirty days of receiving the Muhlenberg/HCD study. The participants were not given an incentive to participate in the survey.

Any internet survey will fail to reach a significant portion of the voting population because of limited access to online service and sampling frames for email addresses that are not inclusive of all households. A third limitation that is more specific to studies using video clips is the limited access that internet users have to high-speed internet services. While dial-up services could accommodate the visuals we tested, the slow speed of loading the ads may have acted as a substantial deterrent to completion of the survey instrument.

With these limitations are recognized, the samples we used in our studies do maintain characteristics that reflect the broader voting population, thus providing a generally representative group of voters on whom to test the advertisement. Sample size and response rates are provided in Table A.

TABLE A

SAMPLE SIZE AND RESPONSE RATES BY ADVERTISEMENTS

  To maintain the integrity of the sample, a number of procedures were utilized. First, we employed password protection to limit access to the survey instrument. Each invitation to participate in the survey contained a password that was uniquely assigned to that email address. A respondent had to enter the password at the beginning of the survey to gain access to it. This process ensured that a respondent completed the survey only once. To increase the number of respondents in the survey and to improve overall response rates, respondents were presented with a summary of selected survey responses.

While our sample was clearly not a random representation of the nation's voting public, the characteristics of survey respondents shared a number of similarities with America's electorate. Table B identifies a number of key demographic qualities of our sample of voters in comparison with a profile of voters in the 2004 election gathered from national exit polling.

TABLE B

A Comparison of Sample and National Voter Demographics

Perhaps the greatest concern with online samples is the exclusion of certain populations from the study, Most notably, online users tend to be younger than the population in general and, more particularly, younger than voters in the United States. As can be seen in Table A, our sample was, indeed, younger than the voting population, with a particular undersampling of individuals sixty years old or over. Our sample also contained a greater percentage of women than the broader voting population, and it underrepresented minority groups.

While acknowledging that our sample varied considerably from the voting population, we can cite some indicators that it generally reflected a number of key political attributes of American voters during the 2004 election season. First, in terms of party identification, our internet-derived sample closely approximated the partisan affiliation of the general voting population. The sample in our study, while overrepresenting liberals, was also generally reflective of the distribution of ideological perspectives among voters. The full distribution is detailed in Table C.

TABLE C

A Comparison of Political Characteristics Between the Survey Sample and Overall Voting Population

Finally, our sample appeared to compare tightly in terms of voter preferences in the 2004 presidential campaign. More specifically, we compared the candidate preferences of survey respondents with results from major national polls taken throughout the election season of 2004. As can be seen in Table D, those of our study were fairly similar to the results of national surveys taken at similar points in time. The differences in the percentage of undecided voters was likely attributable to differences in handling of leaners.

TABLE D

A Comparison of Voter Preferences