AI Governance Engineer
GitHub

Calibration

By Updated

The property that a model's confidence matches its accuracy: of the cases scored 0.9, about nine in ten are right. Modern neural networks are often poorly calibrated 1, so calibration is measured in the eval gate per version and subgroup before any threshold is trusted.

Developed in
ch. 11, Calibration before thresholds
Chapters
ch. 11, AI Defined
Contrast with
Calibration within groups
Source
1 numbered reference, listed below

Where it is used

5 chapters of the Body of Knowledge use the term. Each link opens the first section that does.

Patterns that use this term

3 pattern pages use the term, most mentions first.

  • Conformal prediction

    A distribution-free method that turns a trained model's output into a set of candidate answers that contains the right one with a chosen probability.

  • Predictive AI

    AI that outputs an estimate about something that exists: a score, class or forecast.

  • Eval gate

    A pipeline stage that fails the build when an eval fails; the mechanism that turns an evaluation into an enforced control rather than a report.

Sources

  1. [1] On Calibration of Modern Neural Networks (modern networks poorly calibrated; ICML 2017; arXiv 1706.04599). Guo, Pleiss, Sun and Weinberger. 2017-06-14. https://arxiv.org/abs/1706.04599 (verified: primary)

Definitions of legal terms paraphrase the cited text, which governs. Dated statements are as of .

Cite this term

García Aibar, J. (2026). Calibration. In AI Governance Engineering: The Thesis & Body of Knowledge (v0.5.0), Glossary. https://doi.org/10.5281/zenodo.22956197. https://aigovernanceengineer.com/glossary/calibration. CC BY 4.0

BibTeX

@misc{aige2026calibration,
  author  = {Jorge García Aibar},
  title   = {{Calibration}},
  note    = {Glossary, AI Governance Engineering: The Thesis \& Body of Knowledge, version 0.5.0},
  year    = {2026},
  doi     = {10.5281/zenodo.22956197},
  url     = {https://aigovernanceengineer.com/glossary/calibration}
}