Skip to content
Main Site News Console

FairStress: In-Depth Evaluation of Decision Fairness in Large Language Models

· Hugging Face Translated
教程模型卡

📄 Technical Report | 🤗 Data | 💻 Code Figure 7: FairStress illustration featuring a balance scale surrounded by icons representing hiring, lending, healthcare, education, and housing.

Today, we are officially introducing FairStress, a benchmark designed to evaluate demographic fairness in large language models (LLMs) that looks beyond the final decisions models make in consequential, human-centric scenarios. FairStress is available for download today, and we are also integrating it into Olmo-eval—our open-source platform for running and comparing model evaluations.

Models are increasingly deployed in high-stakes domains where their outputs can directly impact people’s access to opportunities, resources, and care—ranging from screening job applicants and approving loans to triaging patients and evaluating college admissions. Standard demographic fairness benchmarks typically test whether a model selects the same candidate across identical scenarios where only the candidates’ demographic identities are swapped. For instance, if one job applicant has clearly superior experience and credentials, does the model consistently select that candidate, holding qualifications constant, whether they are portrayed as Black in one version or White in another?

However, arriving at the same answer across two versions of a scenario can obscure a critical disparity: how easily the model is swayed to overturn that decision when challenged. Suppose Candidate A is clearly stronger, and the model selects Candidate A regardless of whether A is Black or White. On the surface, this appears fair. Yet if the model is subsequently presented with a persuasive counterargument—such as an executive favoring Candidate B, or Candidate B belonging to a historically underrepresented group—it may abandon the stronger candidate far more readily in one version of the scenario than in the other.

FairStress is designed to measure these hidden asymmetries.

Across open-weight and frontier models alike—including the Olmo, Llama, Qwen, Gemma, Claude, and GPT families—and across various training stages, we find that identical persuasive arguments (which we term adversarial prefixes) make models more or less likely to reverse their decisions depending on which demographic group the argument favors. We also uncovered disparities in how models explain decisions involving demographic information, as well as differences in their internal activation patterns across domains such as hiring, lending, healthcare, education, and housing.

With FairStress, our goal is symmetric fairness: when evidence supports a given decision, the model should apply the same standards and demonstrate equal resistance to reversing that decision, regardless of which demographic group benefits. This reflects a specific, equality-centric definition of fairness: all else being equal, individuals from different groups should have an equal probability of being selected; when one candidate is clearly the superior choice, identity should not alter that outcome; and well-supported decisions should exhibit consistent robustness against adversarial prefixes regardless of which group stands to gain.

FairStress does not attempt to capture every definition of fairness. For instance, equity-based frameworks might treat historical discrimination or underrepresentation as relevant decision factors, potentially favoring minority candidates even when all other qualifications remain equal.

Measuring Fairness Beyond the Final Answer

FairStress comprises 300 realistic, two-candidate decision scenarios spanning 29 domains, including hiring, healthcare, lending, education, housing, and criminal justice. For example, one scenario asks the model to choose between two applicants for a mechanical engineering position, while another involves deciding which of two patients should receive the last available dose of a scarce medication.

A subset of FairStress scenarios features a clearly superior candidate whose profile better satisfies the established criteria. This allows us to test whether demographic information affects how consistently the model selects the candidate most supported by the evidence. Other scenarios are intentionally designed to be ambiguous—where the two candidates