SyRI: a fraud risk model no court could verify
A Dutch court struck down the SyRI fraud-detection legislation in 2020 because the system was insufficiently transparent and verifiable.
One incident read against the controls of AI governance and its frameworks.
- Year
- 2020
- Jurisdiction
- Netherlands
- Sector
- Public sector: welfare and fraud detection
- Evidence base
- Primary sources
- Incident record
- AIAAIC Repository
- Harm
- Unequal treatment of a group by public risk profiling
What happened
SyRI (System Risk Indication) was a legal instrument the Dutch government used to detect fraud with social benefits, allowances and taxes. Data from participating public bodies were linked, pseudonymised and checked against a risk model, and matches produced risk reports on individuals 1.
On 5 Feb 2020 The Hague District Court held that the SyRI legislation did not comply with Article 8(2) of the European Convention on Human Rights: it did not strike the fair balance the Convention requires, and the application of SyRI was insufficiently transparent and verifiable. The court declared the legislation to have no binding effect 1.
The court noted that the State had not given it objectively verifiable information on the nature of SyRI, and that SyRI was used to investigate neighbourhoods known as problem areas 1. The AIAAIC Repository records the case 2.
Failure mode
Opacity was the design, not a side effect. The indicators and the risk model were not open to inspection, so neither the people flagged nor the court could check why a record matched. A control that cannot be inspected cannot be shown to be proportionate.
Deployment by neighbourhood meant the population scanned was chosen before any individual suspicion existed.
Which control would have caught it
The missing artefact is an inspectable description of the model: its purpose, indicators, validation and limits, kept as control evidence rather than as a brochure. Paired with a fundamental-rights impact assessment that states the necessity and proportionality reasoning, and with evidence emitted in a machine-readable form, it gives a court or an auditor something to verify without publishing the model for gaming.
Patterns: Model Card as Control Evidence · FRIA-as-Code · Machine-Readable Evidence (OSCAL)
The evidence that would have existed
What an auditor could have read, and the stack layer that produces it.
- L2 Model card stating purpose, indicators, validation results and known limits, versioned with each risk model
- L1 FRIA recording the interference with private life, the necessity and proportionality reasoning, and how target areas were chosen
- L5 Per-run record of which data sources were linked, how many records were flagged and on which indicators
- L5 Machine-readable evidence package an oversight body can query without access to the raw data
Obligations it touches today
As of 2026-09-24. Mappings are illustrative, not a claim of conformity.
- ECHR Art. 8 The judgment's legal basis: an interference with private life must be transparent and verifiable enough to be weighed 1.
- GDPR Art. 22 Automated individual decision-making, including profiling, is restricted and carries safeguards 7.
- EU AI Act Annex III, point 5(a); Art. 27 Fraud-risk scoring of benefit recipients by public authorities falls in the benefits use case 4; Annex III obligations apply from 2 Dec 2027 (as of 2026-09-24) 5, and public deployers carry out a FRIA before first use 6.
- EU AI Act Art. 86 People affected by a decision based on a high-risk system have a right to an explanation of individual decision-making 3.
Sources
- [1] NJCM et al. v. The State of the Netherlands (SyRI), C/09/550982 / HA ZA 18-388 (ECLI:NL:RBDHA:2020:1878, English translation of ECLI:NL:RBDHA:2020:865). Rechtbank Den Haag (The Hague District Court). 2020-02-05. https://uitspraken.rechtspraak.nl/details?id=ECLI:NL:RBDHA:2020:1878 (verified: primary)
- [2] SyRI welfare fraud detection automation. AIAAIC Repository. 2026. https://www.aiaaic.org/aiaaic-repository/ai-algorithmic-and-automation-incidents/syri-welfare-fraud-detection-automation (verified: primary)
- [3] EU AI Act Art. 86 (right to explanation of individual decision-making based on the output of a high-risk AI system). Publications Office of the EU (EUR-Lex). 2026-07-27. https://eur-lex.europa.eu/eli/reg/2024/1689/2026-07-27/eng#art_86 (verified: primary)
- [4] EU AI Act Annex III (high-risk uses; point 3(b) evaluating learning outcomes, 4(a) recruitment and selection, 5(a) eligibility for essential public assistance benefits and services). Publications Office of the EU (EUR-Lex). 2026-07-27. https://eur-lex.europa.eu/eli/reg/2024/1689/2026-07-27/eng#anx_III (verified: primary)
- [5] AI Omnibus enters into force (Reg. (EU) 2026/1744, in force 2026-07-27; Annex III high-risk obligations move to 2 Dec 2027). European Commission. 2026-07-27. https://digital-strategy.ec.europa.eu/en/news/ai-omnibus-enters-force (verified: primary)
- [6] EU AI Act Art. 27 (FRIA before first use by deployers that are bodies governed by public law or private entities providing public services, and by deployers of Annex III point 5(b) and (c) systems; Art. 27(4) cross-reference to a GDPR Art. 35 DPIA). Publications Office of the EU (EUR-Lex). 2026-07-27. https://eur-lex.europa.eu/eli/reg/2024/1689/2026-07-27/eng#art_27 (verified: primary)
- [7] Regulation (EU) 2016/679 (General Data Protection Regulation) (Art. 5 principles, Art. 6 lawfulness, Art. 8 child's consent, Art. 9 special categories, Arts. 12-15 transparency and access, Art. 22 automated individual decision-making, Art. 33 breach notification, Art. 35 DPIA). Official Journal of the European Union (EUR-Lex). 2016-04-27. https://eur-lex.europa.eu/eli/reg/2016/679/oj/eng (verified: primary)