Standardized tests are widely used to screen candidates, operating under the assumption that they accurately assess innate abilities, often with high-stakes consequences. While significant research has explored how testing procedures can induce performance gaps across groups, little attention has been given to the role of textual content in test items as a potential trigger for these disparities. This paper examines how performance gaps related to socioeconomic status (SES), gender, and ethnicity are influenced by the contextual features of items in the Brazilian admission test, ENEM. The analysis is conducted in two steps. First, I investigate the aggregate performance gaps observed for each group over a 13-year period and over 3.8 million test-takers, analyzing how these gaps relate to the textual content of each item. Using text-analysis techniques, regularization regressions, and assistance from ChatGPT, I develop six testable hypotheses—two for each group: one predicting a widening of gaps and the other predicting a reduction, based solely on the text content. Interestingly, although these hypotheses were generated through a data-driven approach, they align with the well-established Stereotype Threat theory. In the second part, I test these hypotheses using the within-individual data structure. The findings provide strong evidence that the SES gap widens when items include f inancial concepts, with this effect being particularly pronounced among top performers. For gender gaps, the widening effect is driven by items featuring abstract scientific concepts and measurements, though this negative impact can be mitigated when the item also includes a female character. The evidence for gap reduction is weaker in both cases, and no text-induced biases are found for ethnic disparities. These results offer valuable insights for test design, providing a data-driven approach that can be extended to other contexts.

For Information contact angela.baldassarre@unibocconi.it or giulia.zenoni@unibocconi.it.