As anyone who involved in survey research can tell you, one of the more frustrating comments you can get at talks is when people tell you that you should have asked one of your questions differently. While often the questioner has a good point, the fact that you’ve already run the survey introduces a minor time-inconsistency problem: the best time to get this kind of feedback is _before_ the survey goes into the field. However, most of the time when someone wants you to give a talk (or when you are applying to present at a conference), people want to see actual results, not research design. The “American National Electoin Study”:http://www.electionstudies.org/ (ANES) has made a nice step in this direction by setting up an “online commons”:http://www.electionstudies.org/onlinecommons.htm where users can offer comments about the design of proposed new question for the ANES. Lacking the vast resources of the ANES, I figured I’d turn instead to the one resource I did have at my disposal — this blog and its excellent readers — to see if anyone had any suggestions/critiques for an experiment I’m trying to tweak.
The experiment is designed to assess the relative importance of _corruption_ in influencing vote choice as opposed to concerns about the state of the economy. Following the economic voting literature, we^*^ conceive of “corruption voting” as having both potential pocketbook components (e.g., what is the effect of having actually had to pay bribes on your voting behavior?) and soicotropic components (e.g., what is the effect of one’s perceptions about the pervasiveness of corroption on one’s voting behavior?). In a paper that we will be presenting at the American Political Science Association’s Annual Meeting in Seattle a few weeks (Saturday, Sept 1 at 4:15 in Convention Center Room 308), we will report results from a traditional survey in one post-communist country showing much stronger support for “pocketbook corruption voting” than “sociotropic corruption voting”, both in terms of turnout and in terms of voting against the incumbent (i.e., having had to pay a bribe makes you less likely to turnout and less likely to vote for the incumbent). Given the predominence of sociotropic explanations in the economic voting literature _and_ the received wisdom in post-communist countries that part of the reason incumbent governments get voted out of office so often is because people are fed up with overall levels of corruption, this was a pretty surprising finding.
We are now in the process of attempting to see if we can replicate this finding using an experimental research design. We’ve run the experiment once in a different post-communist country and have found results that again suggest pocketbook corruption concerns have more of an effect on voting than sociotropic corruption concerns. However, we have the opportunity to run this experiment in a second country, and we’re hoping to improve the experimental design. As I fear the vast majority of readers of even The Monkey Cage may not be interested in the nuts and bolts of experimental research design, the actual design of the experiment and the question I have about it are found after the break:
Here’s the experiment as designed, with the name of the country XXX’d out:
As you can see the “treatment” for the state of the economy is substantial: a shift from a worsened economy to an improved economy. The “treatment” for corruption, however, is much weaker: from a personal corruption experience (to the extent possible in a survey experiment…) to a perception of corruption. While the survey design allowed us to analyze the relative importance of this pocketbook vs. sociotropic corruption in influencing the voting decision, it is much less useful for analyzing the relative importance of corruption vs. economic conditions from an experimental standpoint because the strength of the treatment is so different.
So here’s one way to think of it: the economic treatment essentially goes from bad to good; the corruption treatment from one form of bad to another form of bad. So for the next version of the experiment, we want to add a “good” treatment to the corruption part of the equation as well. We will have 600 respondents, so we can make this a 3 X 2 experiment and still have 100 respondents in each cell. Here’s how we are thinking of designing the revised version of the experiment – everything will remain the same, but there will be three treatments instead of two. So it will read:
INSERT1 = “Last month, [NAME] had to spend half of his monthly salary to speed up the approval of permits for his business”
INSERT2 = “Last month, [NAME] heard that several city officials have taken bribes in exchange for government contracts”
INSERT3 = “[NAME] lives in a town with a mayor who has a strong reputation for fighting against corruption”
Any thoughts on “INSERT3”? (We’re going to keep the first two the same for comparative purposes.) Does it seem like a satisfactory “good” comparison in terms of corruption? Any concerns with it?
The kicker is that it would be most helpful if I heard back from you in the next 24 hours or so, but any feedback is of course useful. Feel free to respond in comments below or email me directly at joshua dot tucker at nyu dot edu.
Thanks!
************************
^*^ My co-authors on the corruption paper are “Marko Klasnja”:https://files.nyu.edu/mk3296/public/ and “Kevin Deegan-Krause”:http://www.la.wayne.edu/polisci/kdk/. My co-PI on the survey project is “Ted Brader”:http://www.lsa.umich.edu/polisci/people/ci.braderted_ci.detail.