Afghan Polling and What It Actually Means

A recent article from the NYtimes http://www.nytimes.com/2013/12/29/world/asia/polling-comes-to-afghanistan-suggesting-limit-to-sway-of-president-karzai.html?pagewanted=2&_r=0&smid=tw-nytimesworld&partner=rss&emc=rss suggests that Karzai’s power to influence the next elections is limited.
When I read this article, a number of things strike me.
First, why is the US Department of State engaged in political polling for another country? Is it at all possible that this is inappropriate? I would love to go ask folks in Afghanistan what they think about the western practice of political polling. Maybe, they don’t care; of course, that leads to the question of, “why spend the money to do polls in the first place…but we will let that go for the moment. The main question here is, “Is it (culturally?) appropriate for the US DoS to hire a company to conduct a poll, and publish the results of a survey that has the potential to influence elections in a foreign country?”
Let’s test this issue against my “How would we respond if something similar happened to us?” lens. I find this to be a useful way to determine whether or not an activity is culturally offensive, and hence, counterproductive.
Imagine if a foreign nation’s ministry or department of state polled people in New York City regarding our next presidential election, would we see this as benign, benevolent or helpful? Let’s add some detail and say it’s an Islamic country. Further, let’s say this country is occupying our nation. Hmm, I suppose we’d have a problem with that.
Would the people using the information, drawn from a sample of New York City residents, even have the knowledge that the rest of the country might not respond to the same poll in the same manner?
One last thing about the DoS and it’s polling. My personal experience working near the DoS folks is they lack the ability to know what the “people” think. They usually make decisions in a vacuum and tend to disregard the people they are seeking to serve.
This is a critical statement, but I’ve seen on any number of occasions large scale decisions, assessments and plans being worked without the presence of an Afghan. The “Accountability Ladder” of DoS is culturally ignorant and often times offensive, even dangerous, to the people the DoS seeks to help.
Second, Glevum Associates. How do I say this succinctly? I don’t trust anything they produce. My direct experience with Glevum has shown a serious lack of credible information being collected by this organization. One example should suffice…We requested a survey for the district I was researching. Keep in mind, I had previous experiences with Glevum in Iraq that made me reluctant to use their data. This time, when we received our data, I laughed. Glevum Associates had managed to survey more people than the reported population of the district. Again, they found more people than actually exist in this district.
I understand through direct experience that working in conflict zones is basically impossible to do well. I’ve learned to never trust my internal assessments. I constantly challenge what I think, always seeking to eliminate assumptions that I have made along the way. This critical view has allowed me to slow my pace and triple check my work, which might seem tedious, but it has invariably served me well. In this line of work we will make mistakes. I can forgive Glevum for succumbing to the challenges of Afghanistan. What I do not forgive is their arrogance.
When my partner and I brought our concerns to the folks at Glevum, we were dismissed. When fair critical questions regarding research methods were raised, no answers were found. I witnessed an exchange between to PhD level researchers and the doctor presenting Glevum’s research lost badly. Their guy had no real answers for their methods and results.
In a place as challenging as Afghanistan, where data is at best unreliable, one can never assume to have the right answers. There are too many factors that can influence results. This environment demands greater rigor than one would apply to US based research.
Glevum Associates claim a 2% margin of error. That exceeds the level of accuracy of nearly all of our polling data from our last election. Glevum wants you and the DoS to believe their numbers are actually better than numbers we generate here in the US…ugh.
From the article,
“Among the 2,148 likely voters surveyed by Glevum, 85 percent said they would not be swayed if Mr. Karzai decided to endorse a candidate or that it would not matter. The poll, conducted through face-to-face interviews and obtained ahead of its release on Sunday, has a margin of sampling error of plus or minus about two percentage points.”
Let’s apply my critical lens again…if a research company sampled 2200 people in NYC, would any research firm dare to say they were able to report accurately within a 2% margin? No, they wouldn’t.
I think my observations raise some serious questions for our folks at DoS. Who is responsible for requesting this data? Who is holding Glevum accountable for their work? Who is going to have Glevum open the books, for starters, on their collection methodology? Who can truly test the reliability of the information they are providing? A closer look might reveal that their data shouldn’t be used for anything other than an undergraduate statistics assignment on generating descriptive statistics from a convenience sample.
Now with the challenge proposed, will Glevum accept? Will DoS? To be continued…

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s