---
title: "Test-set contamination"
description: "The presence of evaluation items in a model's training data, which inflates its scores; it can be demonstrated even for black-box language models."
canonical: https://aigovernanceengineer.com/glossary/test-set-contamination
author: "Jorge García Aibar"
license: "CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/)"
doi: https://doi.org/10.5281/zenodo.22956197
version: "0.5.0"
updated: 2026-09-24
---

# Test-set contamination

The presence of evaluation items in a model's training data, which inflates its scores; it can be demonstrated even for black-box language models [1]. Mitigated with private held-out sets, rotated items and dated test items.

- Developed in: [ch. 14, Statistical validity of evals](https://aigovernanceengineer.com/bok/governing-development#statistical-validity-of-evals)
- Chapters: [ch. 14, Development](https://aigovernanceengineer.com/bok/governing-development)
- In the glossary chapter: https://aigovernanceengineer.com/bok/glossary#t-test-set-contamination

## Sources

[1] Proving Test Set Contamination in Black Box Language Models (Oren et al.; arXiv 2310.17623). arXiv. 2023-10-26. https://arxiv.org/abs/2310.17623 (verified: primary)