Last week I took aim at Glevum and associates for their questionable collection methodology and subsequent claims of accuracy.
I wanted to take some time to expand on that topic and reveal some of the hazards of collecting data in a combat zone.
Let me start with access, people, and access to people. In the US, it’s completely normal to have several TVs and phones in a household. These things work well for spreading information, enhancing connectivity to a greater social network within the US and ultimately to a much larger global community.
In the context of high information availability and high connectivity to broader social networks, regarding the ability to formulate opinions about relevant political and social issues, and communicate those preferences to survey researchers, citizens of the industrialized world are in a much different position than citizens of developing nations. Not only do citizens of different nations across the planet live in different contexts, many Afghans live in what more closely resembles the Stone Age, but research conducted in different nations occurs in different contexts as well.
Here in the US, we have the ability to use sophisticated methodologies to collect data on individuals that can be analyzed with a high level of statistical certainty, and used to make fairly accurate projections about political matters. This simply is not the case in Afghanistan, which as a “research environment,” poses a few more data collection problems than conducting opinion research in the US. Proper collection is important because improperly collected data adds more error to predictive models of human outcomes that, because they try to predict what humans will do, are likely already full of error.
To help illustrate my point, let’s consider a district (think county government) called Kaki-Afghan also known as Karkar district in Zabul Province (think state government). On a military map this district is “grayed out”…, kinda like on a computer when you don’t have access to something. It’s basically an entire district that is not accessible, and a district about which little is known to the US Military and ISAF. It is both geographically and politically isolated.
It would be nice if someone could answer for me a question, “If the US military will not, or cannot, go somewhere in southern Afghanistan, how does anyone from a survey research organization go there and conduct legitimate public opinion research?” Perhaps there are government contractors doing “socio-cultural” research in contingency environments that have better access than the Department of Defense, but this seems rather unreasonable to me. When it comes to assessing the progress made in a country, realize that entire districts are left out of that equation.

Let’s travel a little to the south and west of that district, to a place called Arghandab district (there are more than one of these in Afghanistan. In this case I am referring to Argendab, Zabul). In this district there are essentially no service members. When there was a small unit there, they had maybe a 1kilometer bubble of movement around their camp. That’s it…beyond that it became too costly, dangerous and problematic to move freely. This risk also applies to the residents of the district. It is far too hazardous for many district residents to travel to the government offices. My question, “How does a civilian research collector go to places an entire US military unit can’t?” In this case what? Most of the populace remains out of reach of the Afghan government/police…and firmly within the range and influence of Taliban fighters.
Let’s discuss those civilian researchers. One of the problems that the “International Partners” face is filling jobs in remote, dangerous areas with qualified applicants. This is due partly to the lack of qualified workers. The outlying regions (i.e. most places outside of Khandahar and Kabul) simply do not contain people with the level of education that are unemployed, underemployed, or basically just willing and able to qualify, apply, interview, and obtain a job. What further complicates the problem of a shortage of good workers is the perceived and actual threat involved with working for the Afghan government (GIRoA).
The Taliban is firmly entrenched into the “infrastructure of the mind” of rural Afghans, so siding with the current government, and making a public statement by taking a public job, is not a decision one takes lightly. A person must have a high level of protection to accept a government job. Protection can take many different forms, but the point is that Afghans must be accept a level of risk when accepting these positions.
Adding to this complex mix of problems, there are educated and qualified Afghans from Kabul and Kandahar….but due to perceptions (both real and unfounded), the more “urban” Afghans are reluctant to “venture to the hills” for the purpose of working for the current government in the back country and distant districts of their country. Just like rural “Afghans” have their own perceptions about the power of their country’s more extreme political elements embedded into the infrastructure of their minds, we should not assume that qualified “urban” Afghans do not factor the threat of political violence into their decisions to seek or not seek out a government job. We must always remember that instability (read “fear of death”) is closer to the norm there than stability (read “certainty of life”). As it is, instability causes villages and tribes to bond tightly, and avoid extending trust to outsiders. I need to take a time for an aside to add some context to this conversation.
Villagers in districts are constantly encountering Taliban fighters. Not necessarily in a combative nature, but in daily life. What this means is that extremist elements act with impunity in most of the rural districts, especially in the Southern region. In reality, these Taliban fights are the government. The police rarely, if ever, arrest these individuals, cases of kidnapping are never solved or even reported, etc., etc.,.
Given the level of perceived and real threat, how does a company that contracts with the US government to conduct polls in combat environments send out researchers into the field? What compels their researcher to take on this task? Is it money? Since they are going alone, how do we actually know if we can trust their collection protocol? Is data being fabricated? What steps do Glevum Associates take to protect their collectors? More importantly, what steps were taken to protect the well-being of the subject interviewed in their polls? In distant villages, strangers are obvious to residents, thus hindering the ability of survey researchers to properly sample a given population, employ a survey instrument, and accurately assess public opinion with techniques identical to what is commonly used in the US.
My question to Glevum Associates, how do you overcome these obstacles? How do you ensure the data is accurate? How do you protect your researchers and your subjects?
Finally, the geography and environmental conditions in Afghanistan are such that travelling takes an enormous amount of time. Assuming a permissive environment, the act of walking village to village, (with no maps or GPS) for a stranger (read foreign or domestic, not local) to conduct a survey is a relatively resource intensive endeavor, one so drastic that the military finds it to be too challenging, and therefore hires a company like Glevum to do it for them, and somehow, Glevum appears to get information from places that even the US military will not go.
Let’s remember the organizations that do field research for the US are driving decisions and compiling data that is used to make decisions. I submit that a large portion of the data is suspect. We can do better than this. If we are going to be involved in conflict zones, our partners expect more from us.