What Can Experimental Studies of Bias Tell Us About Real-World Group Disparities?

This article questions the widespread use of experimental social psychology to understand real-world group disparities. Standard experimental practice is to design studies in which participants make judgments of targets who vary only on the social categories to which they belong. This is typically done under simplified decision landscapes and with untrained decision makers. For example, to understand racial disparities in police shootings, researchers show pictures of armed and unarmed Black and White men to undergraduates and have them press "shoot" and "don't shoot" buttons. Having demonstrated categorical bias under these conditions, researchers then use such findings to claim that real-world disparities are also due to decision-maker bias. I describe three flaws inherent in this approach, flaws which undermine any direct contribution of experimental studies to explaining group disparities. First, the decision landscapes used in experimental studies lack crucial components present in actual decisions (Missing Information Flaw). Second, categorical effects in experimental studies are not interpreted in light of other effects on outcomes, including behavioral differences across groups (Missing Forces Flaw). Third, there is no systematic testing of whether the contingencies required to produce experimental effects are present in real-world decisions (Missing Contingencies Flaw). I apply this analysis to three research topics to illustrate the scope of the problem. I discuss how this research tradition has skewed our understanding of the human mind within and beyond the discipline and how results from experimental studies of bias are generally misunderstood. I conclude by arguing that the current research tradition should be abandoned.