---
title: "A recruiting model that learned the past (reported)"
description: "Reuters reported in 2018 that an experimental recruiting model trained on a decade of mostly male CVs learned to downgrade women; the project was dropped."
canonical: https://aigovernanceengineer.com/cases/recruiting-model-reported
author: "Jorge García Aibar"
license: "CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/)"
doi: https://doi.org/10.5281/zenodo.22956197
version: "0.5.0"
updated: 2026-09-26
---

# A recruiting model that learned the past (reported)

> Reuters reported in 2018 that an experimental recruiting model trained on a decade of mostly male CVs learned to downgrade women; the project was dropped.

- Year: 2018
- Jurisdiction: Not stated in the reports
- Sector: Employment: recruitment
- Evidence base: Reported
- Incident record: [AIID 37](https://incidentdatabase.ai/cite/37/)
- Harm: [Discrimination in consequential decisions](https://aigovernanceengineer.com/resources/harms#harm-discriminatory-decisions)

## In short

Reuters reported on 10 Oct 2018 that Amazon built an experimental tool to score job applicants, trained on about ten years of CVs drawn largely from men. It reportedly learned to penalise CVs with the word "women's" and graduates of certain all-women colleges, and the project was abandoned. These are reported facts: no primary company document is public. The harm is discrimination in a consequential decision: hiring. The failure mode is historical label bias: the model learned what past hiring looked like, and removing gendered terms does not remove their proxies. An Eval Gate in CI computing selection rates by sex and failing the build below an impact-ratio threshold, Model Card as Control Evidence with a data card on the training population, and FRIA-as-Code on the decision to automate screening would have caught it early. The case touches EU AI Act Annex III point 4(a), which lists recruitment as high-risk, Art. 10 on training-data bias, and the bias audits of NYC Local Law 144.

## What happened

Reuters reported on 10 Oct 2018 that Amazon had built an experimental tool, from 2014, to score job applicants; that it was trained on about ten years of CVs drawn largely from men; that it learned to penalise CVs containing the word "women's" and graduates of certain all-women colleges; and that the project was abandoned [1][2]. These are reported facts: no primary company document is public.

The company is reported to have said that recruiters never relied solely on the tool's rankings [2].

## Failure mode

Historical label bias. The model learned what past hiring looked like, past hiring skewed male, and the skew became a scoring rule. Removing explicit gendered terms does not remove proxies for them, which is why reports say the fixes did not guarantee fairness [2].

## Which control would have caught it

An eval gate that computes selection rates by sex (and other protected characteristics) on a held-out set, and fails the build when an impact ratio falls below a set threshold, catches this in the first release candidate rather than after years of development. A data card that states the composition of the training population makes the risk visible before training starts. The fundamental-rights impact assessment is where the decision to automate screening at all is argued.

Patterns: [Eval Gate in CI](https://aigovernanceengineer.com/bok/patterns#pattern-eval-gate-in-ci) · [Model Card as Control Evidence](https://aigovernanceengineer.com/bok/patterns#pattern-model-card-as-control-evidence) · [FRIA-as-Code](https://aigovernanceengineer.com/bok/patterns#pattern-fria-as-code)

## The evidence that would have existed

What an auditor could have read, and the stack layer that produces it.

- Layer 2 (Inventory & Transparency): Data card stating the composition of the training population by sex and period
- Layer 3 (Evals & Red Teaming as Evidence): Bias-audit report per release with selection rates and impact ratios by group
- Layer 3 (Evals & Red Teaming as Evidence): Eval-gate log showing the release blocked or passed on those ratios
- Layer 1 (Govern-as-Code): FRIA recording the decision to automate screening and its safeguards

## Obligations it touches today

As of 2026-09-24. Mappings are illustrative, not a claim of conformity.

- EU AI Act Annex III, point 4(a): Recruitment and selection, including filtering applications and evaluating candidates, is high-risk [3]; Annex III obligations apply from 2 Dec 2027 (as of 2026-09-24) [4].
- EU AI Act [Art. 10](https://aigovernanceengineer.com/obligations/aige-obl-euaia-art10): Training data for high-risk systems must be examined for possible biases [5].
- New York City [Local Law 144 of 2021](https://aigovernanceengineer.com/obligations/aige-obl-usnyc-ll144): Employers may not use an automated employment decision tool unless it has had a bias audit within one year, the audit information is public and notices have been given; enforcement began on 5 Jul 2023 [6].

## How to read this case

Each case is an illustrative engineering analysis of public records, not a legal determination, not a finding of fact beyond what the cited sources state, and not a claim of conformity. Mappings to obligations are illustrative.

## Sources

[1] Amazon scraps secret AI recruiting tool that showed bias against women. Reuters. 2018-10-10. https://www.reuters.com/article/us-amazon-com-jobs-automation-insight-idUSKCN1MK08G (verified: reported)
[2] AI Incident Database, Incident 37: Amazon's Experimental Hiring Tool Allegedly Displayed Gender Bias in Candidate Rankings. Responsible AI Collaborative. 2026. https://incidentdatabase.ai/cite/37/ (verified: primary)
[3] EU AI Act Annex III (high-risk uses; point 3(b) evaluating learning outcomes, 4(a) recruitment and selection, 5(a) eligibility for essential public assistance benefits and services). Publications Office of the EU (EUR-Lex). 2026-07-27. https://eur-lex.europa.eu/eli/reg/2024/1689/2026-07-27/eng#anx_III (verified: primary)
[4] AI Omnibus enters into force (Reg. (EU) 2026/1744, in force 2026-07-27; Annex III high-risk obligations move to 2 Dec 2027). European Commission. 2026-07-27. https://digital-strategy.ec.europa.eu/en/news/ai-omnibus-enters-force (verified: primary)
[5] EU AI Act Art. 10 (data and data governance; examination of training data for possible biases). Publications Office of the EU (EUR-Lex). 2026-07-27. https://eur-lex.europa.eu/eli/reg/2024/1689/2026-07-27/eng#art_10 (verified: primary)
[6] Automated Employment Decision Tools (AEDT) (Local Law 144 of 2021; bias audit within one year of use; enforcement from 5 Jul 2023). NYC Department of Consumer and Worker Protection. 2023. https://www.nyc.gov/site/dca/about/automated-employment-decision-tools.page (verified: primary)