bq. By the same logic, my study of human behavior indicates that Americans of both genders typically wake up between 10 and 11 a.m., and subsist primarily on Natty Light and pizza. Yet somehow everyone understands that college students’ behavior does not allow us to draw generalizable conclusions about human behavior. And yet I’m constantly seeing psychology studies that look at a small sample of college students and draw wildly broad conclusions. College students aren’t even demographically representative of the college-age population. Can’t we do better than this?
We may not need to. Jamie Druckman and Cindy Kam:
bq. In this chapter, we investigate the extent to which using students as experimental participants creates problems for causal inference. First, we discuss the impact of student subjects on a study’s internal and external validity. In contrast to common claims— including Sear’s (1986) widely cited proclamation of students being a “narrow data base”—we argue that student subjects do not intrinsically pose a problem for a study’s external validity. Second, we use simulations to identify situations when student subjects are likely to constrain experimental inferences. We show, perhaps surprisingly, that such situations are relatively limited. Third, we briefly survey empirical evidence that provides guidance on when researchers should be particularly attuned to taking steps to ensure appropriate generalizability from student subjects. We conclude with a discussion of the practical implications of our findings. In short, we argue that student subjects are not an inherent problem to experimental research; moreover, a case can be made that the burden of proof—of student subjects being a problem—should lie with critics rather than experimenters.
They note that the ability to generalize from an experiment involves many dimensions besides the demographics of the experimental participants. To me, experiments are probably compromised more by the way they fail to mimic real-life situations than by their 19-year-old subjects.