Surveys show that Americans do not know a lot of facts about politics (see, e.g., here). But why? A simple answer is that people do not know them.
But in a new paper (here or here, gated) Arthur Lupia and Markus Prior have a more complicated answer. They suggest that people may not give correct answers to knowledge questions because (1) they do not put forth much effort in answering the question or (2) cannot quickly recall the answer but perhaps could easily find it if given the chance. A short survey interview does not tend to encourage much effort; at the same time, it tests only immediate recall.
So Lupia and Prior designed an experiment. Each subject was asked to answer 14 knowledge questions. The experiment had four conditions:
1) A control group, which had 1 minute to answer each question.
2) The “money” group, which had 1 minute to answer each question and was paid $1 for each correct answer.
3) The “time” group, which was given 24 hours to answer these 14 questions.
4) The “money + time” group, which was given 24 hours and $1 per correct answer.
The results? Here is the mean number of correct answers in each group:
Control: 4.5 (out of 14)
Money: 5.0
Time: 5.4
Money + time: 5.6
The differences relative to the control group are statistically significant. Moreover, Lupia and Prior find that money and time are most effective upon those without a college degree and with only moderate interest in politics.
They conclude:
bq. Our results provide a new and distinct reason for being skeptical when analysts use existing knowledge measures as the basis for sweeping generalizations about what citizens do not know…In particular, existing political knowledge measures, when used as measures of political competence, likely underestimate the public’s true abilities.
In thinking about this, it occurred to me that a skeptic (or, better put, a pessimist) might say: If the average respondent in every group answered about 5 or 6 out of 14 questions, is this “sweeping generalization” really that inaccurate? Is most of the variance in knowledge really explained by, well, knowledge, rather than by a lack of effort or ability to recall the answers correctly?
Those interested in this topic should also see this paper by Lupia, which questions the value of these knowledge questions in assessing the public’s competence as democratic citizens.