AI Governance Engineer
GitHub

What AI harms, and what catches it.

Start from a named harm, not a feature list. Each row gives the level it lands on, the mechanism behind it, a failure mode you can test for, the control and stack layer that catch it, the evidence that control leaves, and incidents that show it is not hypothetical.

The atlas

25 harms across five levels, 35 incident records. Read each card as a sentence: this harm happens through this failure mode, is caught by this control in this layer, and leaves this evidence.

Cross-reference: incident cases · the pattern catalogue · the obligation index.

Download JSON

Filter by level

Individual

5 harms

Harm to one person: their rights, money, liberty, autonomy or body.

  • Discrimination in consequential decisions

    • Bias

    A model that scores people for jobs, credit, care or benefits treats a protected group worse at equal merit or need, usually because it learned from past decisions or from a proxy target. In one widely used health-risk algorithm, removing the disparity would have raised the share of Black patients flagged for extra help from 17.7% to 46.5% 6.

    Failure mode
    The training label or a feature is a proxy (past hiring, past spending) that encodes the disparity the decision must not repeat.
    Caught by
    Eval Gate in CI L3
    Evidence
    A disaggregated eval report per release (selection or flag rate and calibration by group) with the gate verdict that passed or blocked the build.
    MIT taxonomy
    • 1.1 Unfair discrimination and misrepresentation (1 Discrimination & toxicity)
    • 1.3 Unequal performance across groups (1 Discrimination & toxicity)
    Incidents
    Cases

  • Wrongful identification and loss of liberty

    • Bias
    • Over-reliance

    A face-recognition match is a ranked probability, not an identification. When investigators act on a match without independent corroboration, a system error becomes the arrest of the wrong person.

    Failure mode
    A candidate list from a probabilistic matcher is treated as a positive identification, with no mandatory corroboration step before action.
    Caught by
    Human-in-the-loop Gate L4
    Evidence
    An approval record naming the reviewer, the corroborating evidence and the decision, written before any action is taken on a match.
    MIT taxonomy
    • 1.3 Unequal performance across groups (1 Discrimination & toxicity)
    • 5.1 Overreliance and unsafe use (5 Human-computer interaction)
    Incidents

  • Privacy intrusion and biometric surveillance

    • Scale
    • Misuse

    Scraping images at internet scale and turning each face into a biometric template exposes people to identification they never agreed to. The Dutch data protection authority fined one such provider EUR 30.5 million and found it should never have built the database 9.

    Failure mode
    Personal and biometric data are collected without a lawful basis, and buyers integrate the resulting service without asking where its data came from.
    Caught by
    Vendor / Model Due-Diligence Gate L2
    Evidence
    A due-diligence record holding the provider's lawful-basis and data-provenance answers, the DPIA reference and the approve or reject decision.
    MIT taxonomy
    • 2.1 Compromise of privacy by obtaining, leaking, or correctly inferring sensitive information (2 Privacy & security)
    Incidents
    Cases

  • Manipulation, dependence and loss of autonomy

    • Misalignment

    Conversational systems tuned for engagement can foster emotional dependence and steer vulnerable users, minors among them, when nothing at runtime recognises the risk. The incident record here is an allegation 3.

    Failure mode
    The system optimises for continued conversation and has no runtime check for self-harm signals, age or escalating dependence.
    Caught by
    Runtime Guardrail L4
    Evidence
    Guardrail decision logs for self-harm and age signals, each with the escalation or hand-off it triggered.
    MIT taxonomy
    • 5.1 Overreliance and unsafe use (5 Human-computer interaction)
    • 5.2 Loss of human agency and autonomy (5 Human-computer interaction)
    Incidents

  • Physical injury from systems that act in the world

    • Robustness

    Automated driving and other embodied systems turn a perception or planning error into bodily harm. The records include a pedestrian killed by a test vehicle and a pedestrian dragged by a driverless taxi 3.

    Failure mode
    The system meets a situation outside its validated envelope and has no safe fallback that stops it or hands over before harm.
    Caught by
    Kill Switch / Circuit Breaker L4
    Evidence
    A tested stop mechanism with drill records, and telemetry showing when the fallback fired and which safe state the system entered.
    MIT taxonomy
    • 7.3 Lack of capability or robustness (7 AI system safety, failures & limitations)
    Incidents

Group

3 harms

Harm that falls on a group or community as such, not only on its members one by one.

  • Unequal treatment of a group by public risk profiling

    • Bias
    • Scale

    Risk models used to police benefits concentrate suspicion on groups defined by nationality, income or neighbourhood, and every false flag lands on a household. The Dutch tax administration used Dutch or non-Dutch nationality as an indicator in a system that designated childcare-benefit applications as risky 8.

    Failure mode
    A protected or proxy attribute enters the risk model, and flags drive adverse action with no group-level outcome monitoring.
    Caught by
    FRIA-as-Code L1
    Evidence
    A versioned fundamental-rights impact assessment naming the affected groups, the permitted features and the monitoring metric, signed before deployment.
    MIT taxonomy
    • 1.1 Unfair discrimination and misrepresentation (1 Discrimination & toxicity)
    Incidents
    Cases

  • Exclusion through under-representation in data

    • Bias

    A group that is thin in the training data gets a worse service. Five commercial speech-recognition systems averaged a word error rate of 0.35 for Black speakers against 0.19 for white speakers 7.

    Failure mode
    Performance is reported as one average, so the gap for a minority group never reaches a release decision.
    Caught by
    Model Card as Control Evidence L2
    Evidence
    A model card with performance disaggregated by the groups the system serves, and a data card describing who is in the training set.
    MIT taxonomy
    • 1.3 Unequal performance across groups (1 Discrimination & toxicity)
    Incidents

  • Judging individuals by their group's history

    • Bias
    • Opacity

    When a model predicts an outcome distribution for a group and then places individuals within it, a person's result depends on the past record of their school, postcode or cohort. England's 2020 grading model predicted each centre's grade distribution from its history and used teachers' rank orders to assign grades within it 13.

    Failure mode
    Acceptance tests are aggregate (distribution accuracy, group equality), so no test asks whether an individual outcome can be justified from evidence about that individual.
    Caught by
    Human-in-the-loop Gate L4
    Evidence
    A review queue and decision log for outlier cases (large adjustments, small cohorts), plus a contestation channel with its resolution records.
    MIT taxonomy
    • 1.1 Unfair discrimination and misrepresentation (1 Discrimination & toxicity)
    Incidents
    Cases

Organisation

9 harms

Risk to the organisation that builds, buys or deploys the system: operational, legal, financial, reputational, security and supply chain.

Society

6 harms

Harm to shared systems: information, elections, work, equality and essential services.

  • Pollution of the information ecosystem

    • Scale
    • Misuse

    Cheap synthetic text, images and audio make it harder to tell what is real, which erodes trust in authentic evidence as well as in fakes. NIST lists information integrity among its twelve generative-AI risks 12.

    Failure mode
    Generated media leave the system with no provenance signal that a platform or a reader could check.
    Caught by
    Content provenance and labelling (not yet a catalogued pattern) L2
    Evidence
    Provenance metadata and labels attached at generation, with the share of outputs that carried them.
    MIT taxonomy
    • 3.2 Pollution of information ecosystem and loss of consensus reality (3 Misinformation)
    • 4.1 Disinformation, surveillance, and influence at scale (4 Malicious actors & misuse)
    Incidents

  • Election interference with synthetic voices and media

    • Misuse

    Synthetic audio of a candidate does most damage just before a vote, when there is no time to rebut it. The records include a fake presidential voice in robocalls before a US primary and deepfake recordings spread before Slovakia's 2023 election 3.

    Failure mode
    Voice cloning is available without a consent check on whose voice is cloned, and the output carries no durable mark.
    Caught by
    Content provenance and labelling (not yet a catalogued pattern) L2
    Evidence
    Consent records for every cloned voice, and watermark or provenance checks on generated audio.
    MIT taxonomy
    • 4.1 Disinformation, surveillance, and influence at scale (4 Malicious actors & misuse)
    Incidents

  • Job displacement and task transformation

    • Scale

    Generative AI changes what many jobs consist of. The ILO estimates that one in four workers globally are in an occupation with some generative-AI exposure, and finds transformation of jobs more likely than outright replacement 10. This harm is structural, so no single incident record stands for it.

    Failure mode
    Deployment decisions count the productivity gain but not who loses tasks, income or bargaining power, and affected workers are not consulted.
    Caught by
    Workforce impact assessment (not yet a catalogued pattern) L1
    Evidence
    An impact assessment listing the roles and tasks affected, the consultation held and the transition measures agreed.
    MIT taxonomy
    • 6.2 Increased inequality and decline in employment quality (6 Socioeconomic & environmental harms)
    Incidents
    No single incident record: the evidence is the study cited above.

  • Inequality in who bears the cost: hidden data labour

    • Scale

    The gains from AI flow to its builders and users while part of the cost falls on low-paid workers who label data, sometimes exposed to graphic material. The record here alleges that annotators in Kenya were exposed to graphic content while labelling for a model developer 3.

    Failure mode
    Data-supply contracts are bought on price with no labour or welfare standard, and nobody audits them.
    Caught by
    Vendor / Model Due-Diligence Gate L2
    Evidence
    A supplier assessment covering annotator pay, exposure limits and support, renewed with the contract.
    MIT taxonomy
    • 6.2 Increased inequality and decline in employment quality (6 Socioeconomic & environmental harms)
    • 6.1 Power centralization and unfair distribution of benefits (6 Socioeconomic & environmental harms)
    Incidents

  • Over-reliance on generated output in institutions

    • Over-reliance

    Professionals under time pressure accept fluent output without checking it. The records include fabricated case law filed in a US court and errors introduced into a child-protection report 3.

    Failure mode
    Generated content enters a consequential record with no verification step and no disclosure that AI was used.
    Caught by
    Human-in-the-loop Gate L4
    Evidence
    A sign-off record showing who verified each generated passage against its source before filing.
    MIT taxonomy
    • 5.1 Overreliance and unsafe use (5 Human-computer interaction)
    • 3.1 False or misleading information (3 Misinformation)
    Incidents

  • Disruption to critical infrastructure and emergency services

    • Robustness
    • Scale

    When AI systems run transport, utilities or emergency response, one failure repeats across a whole fleet at once. The records include driverless cars that stalled in traffic after losing their server connection, and one that blocked a fire truck on an emergency call 3.

    Failure mode
    A shared dependency (connectivity, a model update) fails for every unit at once, and there is no degraded mode that clears the way.
    Caught by
    Kill Switch / Circuit Breaker L4
    Evidence
    Fleet-level stop and safe-state drills, with records of when the breaker fired and how long recovery took.
    MIT taxonomy
    • 7.3 Lack of capability or robustness (7 AI system safety, failures & limitations)
    Incidents

Environment

2 harms

Harm to the physical environment from building and running AI systems.

  • Energy demand and emissions of training and inference

    • Scale

    Training and serving models draws electricity, and so emissions, at data-centre scale. The IEA projects that data-centre electricity demand will more than double by 2030 to around 945 TWh, slightly more than Japan's consumption today 11. NIST lists environmental impacts among the generative-AI risks 12.

    Failure mode
    Model and deployment choices are made without measuring energy use, so the cheapest adequate option is never compared.
    Caught by
    Energy and emissions telemetry (not yet a catalogued pattern) L5
    Evidence
    Per-model energy and emissions figures, recorded for training and per unit of inference, published in the model card.
    MIT taxonomy
    • 6.6 Environmental harm (6 Socioeconomic & environmental harms)
    Incidents
    No single incident record: the evidence is the study cited above.

  • Local pollution and resource burden near AI infrastructure

    • Scale

    The power generation and cooling that AI data centres need can burden the communities around them. The record here alleges that a supercomputer site in Memphis ran unpermitted gas turbines 3.

    Failure mode
    Capacity is added faster than permitting and community consultation, and buyers of compute never ask where or how it is powered.
    Caught by
    Vendor / Model Due-Diligence Gate L2
    Evidence
    The provider's answers on energy source, permits and location, kept with the procurement decision.
    MIT taxonomy
    • 6.6 Environmental harm (6 Socioeconomic & environmental harms)
    Incidents

Sources and attribution

Harm categories use the Domain Taxonomy of the MIT AI Risk Repository (Slattery et al., 2026), licensed under CC BY 4.0: https://airisk.mit.edu/ (https://doi.org/10.1016/j.patter.2026.101517). Codes and names are reproduced unchanged; the mapping of each harm to a subdomain is ours.

  1. [1] The AI Risk Repository: a meta-review, database, and taxonomy of risks from artificial intelligence (Domain Taxonomy of 7 domains and 24 subdomains; published under CC BY 4.0). Slattery, P., Saeri, A. K., Grundy, E. A. C., et al., Patterns (Cell Press). 2026. https://doi.org/10.1016/j.patter.2026.101517 (verified: primary)
  2. [2] MIT AI Risk Repository (the living database of AI risks classified by the Domain Taxonomy; CC BY 4.0). MIT AI Risk Initiative. 2026. https://airisk.mit.edu/ (verified: primary)
  3. [3] AI Incident Database (incident records cited as AIID <number>). Responsible AI Collaborative. 2026. https://incidentdatabase.ai/ (verified: primary)
  4. [4] AIAAIC Repository (AI, algorithmic and automation incidents and controversies). AIAAIC. 2026. https://www.aiaaic.org/aiaaic-repository (verified: primary)
  5. [5] AI Incidents and Hazards Monitor (AIM) (news-derived incident records). OECD.AI. 2026. https://oecd.ai/en/incidents (verified: primary)
  6. [6] Dissecting racial bias in an algorithm used to manage the health of populations (Science 366(6464):447-453; remedying the disparity would raise Black patients receiving additional help from 17.7% to 46.5%). Obermeyer, Z., Powers, B., Vogeli, C., Mullainathan, S.. 2019-10-25. https://doi.org/10.1126/science.aax2342 (verified: primary)
  7. [7] Racial disparities in automated speech recognition (PNAS 117(14):7684-7689; five commercial systems, average word error rate 0.35 for Black speakers and 0.19 for white speakers). Koenecke, A., Nam, A., Lake, E., et al.. 2020-04-07. https://doi.org/10.1073/pnas.1915768117 (verified: primary)
  8. [8] Tax Administration fined for discriminatory and unlawful data processing (EUR 2.75 million fine; nationality used as an indicator in a system that designated childcare-benefit applications as risky). Autoriteit Persoonsgegevens (Dutch Data Protection Authority). 2021-12-07. https://www.autoriteitpersoonsgegevens.nl/en/current/tax-administration-fined-for-discriminatory-and-unlawful-data-processing (verified: primary)
  9. [9] Dutch DPA imposes a fine on Clearview because of illegal data collection for facial recognition (EUR 30.5 million fine plus orders subject to penalties). Autoriteit Persoonsgegevens (Dutch Data Protection Authority). 2024-09-03. https://www.autoriteitpersoonsgegevens.nl/en/current/dutch-dpa-imposes-a-fine-on-clearview-because-of-illegal-data-collection-for-facial-recognition (verified: primary)
  10. [10] Generative AI and jobs: a refined global index of occupational exposure (ILO Working Paper 140; one in four workers globally in an occupation with some GenAI exposure; transformation of jobs the most likely impact). International Labour Organization. 2025-05-20. https://www.ilo.org/publications/generative-ai-and-jobs-refined-global-index-occupational-exposure (verified: primary)
  11. [11] AI is set to drive surging electricity demand from data centres while offering the potential to transform how the energy sector works (Energy and AI report; data-centre electricity demand to more than double by 2030 to around 945 TWh). International Energy Agency. 2025-04-10. https://www.iea.org/news/ai-is-set-to-drive-surging-electricity-demand-from-data-centres-while-offering-the-potential-to-transform-how-the-energy-sector-works (verified: primary)
  12. [12] Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile (NIST AI 600-1; twelve risks including confabulation, information integrity, environmental impacts and value chain and component integration). NIST. 2024-07-26. https://doi.org/10.6028/NIST.AI.600-1 (verified: primary)
  13. [13] Awarding GCSE, AS, A level, advanced extension awards and extended project qualifications in summer 2020: interim report (the Direct Centre Performance model predicted each centre grade distribution from its history, then used teacher rank orders to assign grades). Ofqual. 2020-08-13. https://www.gov.uk/government/publications/awarding-gcse-as-a-levels-in-summer-2020-interim-report (verified: primary)
  14. [14] Zillow Group Reports Third-Quarter 2021 Financial Results; Shares Plan to Wind Down Zillow Offers Operations (Form 8-K, Exhibit 99.1; inventory write-down of approximately USD 304 million; workforce reduction of approximately 25%). Zillow Group, Inc. (SEC EDGAR). 2021-11-02. https://www.sec.gov/Archives/edgar/data/1617640/000161764021000085/q32021991.htm (verified: primary)