| Ethno-linguistic fragmentation. Like Bosnia and Iraq, Afghanistan is ethnically and linguistically fragmented, with substantial and enduring cleavages. The four major ethnic groups are Pashtun, comprising 44 percent of the population; Tajik, 25 percent; Hazara, 10 percent; and Uzbek, 8 percent; and other groups total the remaining 13 percent. While the country is 99 percent Muslim, 84 percent are Sunni and 15 percent are Shia. The official language, Dari, is a unifying force, with 50 percent of the population speaking it. Other key languages are Pashtu (35 percent) and Turkic languages (Uzbek and Turkmen, spoken by 11 percent). Building trust and rapport with respondents requires that interviewers be local and share ethnicity with them. This is especially true outside of Kabul. ACSOR has successfully worked to build a multiethnic, multilingual team to overcome these challenges across Afghanistan.
Outdated and inconsistent population data. The last Afghan census took place prior to the Soviet invasion in December 1979. Since then, war, terrorism, and economic dislocations have caused millions of Afghans to become refugees (largely in Pakistan and Iran) or internally displaced persons. Working with census data from the 1970s and more current data or estimates from the Afghan Census Department, the Agriculture/Irrigation Department, and nongovernmental organizations (NGOs), ACSOR is able to estimate the urban and rural population figures across Afghanistan's thirty-two provinces. We believe we are working with the best available estimates and are able to make updates of our own as our base of surveys continues to grow.
Our Afghan national opinion surveys are based on multistage random sampling, from selection of geographic sampling points through respondent selection. The sampling points are distributed proportionally to the population by region and urban/rural strata with no more than ten interviews per sampling point. The sampling points and actual starting points for interviewers are randomly selected from lists of existing districts in Afghanistan. Interviewers select the households using the random route method, and then the respondents by using a Kish grid. If the designated respondent is not available at the first contact, the field team makes two more attempts in urban areas-but only one more in rural areas due to logistics-before moving on. However, widespread unemployment and use of weekdays and weekends yield a high completion rate of around 90 percent on the first contact, with relatively few interviews requiring a second or third attempt. D3 does not set a quota for gender, but we do focus the field management's attention on the issue and maintain as strong a female contingent in the interviewing force as possible. Afghanistan is really no different than any other Muslim country, where, without strenuous attention to gender balance in the field force and sampling, surveys tend to have a strongly male bias.
Establishing and maintaining a high standard of fieldwork in Afghanistan poses the same challenges as it would anywhere else, along with a number of additional ones. ACSOR evaluates all of the work of new interviewers the first time they go into the field. If the work is satisfactory, the interviewer will then be evaluated on a random basis, but the majority of interviewers have at least some portion of their work checked in each survey. The primary responsibility for field force evaluation, training, and management (dismissal, retention, or promotion) lies with the local research firm and supervisors. They are able to observe some interviews directly, back-check others, and review and edit each questionnaire.
No matter how energetic the observers, with usually over a hundred interviewers working on a national survey covering a large geographic area, it is not possible to monitor work in all locations during a two-or three-week field period. To help identify problem interviewers and recognize those who are performing very well, D3 has developed a set of statistical tests taking variance and nonresponse into account by interviewer. Three of these tests highlight potential problems, and one attempts to explain them.
Test 1 looks for patterns of consistent responses across questions by a respondent. This typically occurs in rating batteries, with the respondent giving the same answer (such as "agree somewhat") to every item, despite random or rotating reading order. Such a pattern suggests a problem with the interviewer, especially if the rated items (political leaders or parties) are disparate.
In some instances, articulate and responsive subjects will refuse to answer a certain type of question, producing a "straight" that the interviewer can do nothing about. Afghan respondents facing possible disapproval from their families will opt out of some sensitive questions rather than disturb the peace; people who were generally talkative on political topics will either refuse or say they do not know when it comes to rating political leaders, parties, or their own likely voting behavior. For this reason, we often distinguish between "substantive straights" (all responses being, for instance, "somewhat agree") and nonsubstantive straights, where the respondent opts out of the question on sensitivity or knowledge grounds. In all cases, the presence of straights is not in itself evidence of failure. We expect some level of patterned responses and focus our attention on those interviewers who exceed the norm for that survey. |