Pattern: Eval Gate in CI

An evaluation suite wired into CI so a model or agent ships only above a documented threshold: the eval run is the control, its result the evidence.

By Jorge García Aibar
v0.5.0
Updated 2026-09-26
CC BY 4.0

Layer 03 · Evals & Red Teaming as Evidence In the chapter 05 catalogue

One working control on the engineering side of AI governance.

Summary: Wire an evaluation suite into the CI/CD pipeline so that a model or agent must pass a defined test, above a documented threshold, before it can ship. The eval run is the control and its result is the evidence; a failing eval blocks the build.

In short

Eval Gate in CI is an evaluation control that wires an evaluation suite into the CI/CD pipeline, so a model or agent must pass a defined test, above a documented threshold, before it ships. It solves one-off pre-launch evaluations pasted into a slide, which prove nothing after the next change, and a review board that can rate findings but not stop a launch. Use it for a model or agent retrained, re-prompted or given new tools, whose pipeline already runs functional tests. The suite, versioned with the model, runs at least one capability and one adversarial eval; each threshold traces to a named failure mode or obligation. A result below it fails the build, and every result is filed against the registry entry, so the eval run is the control and its result the evidence. Its illustrative mappings are EU AI Act Art. 15 and 55, ISO/IEC 42001, the NIST AI RMF Measure function and OWASP Agentic ASI01 and ASI02.

Eval Gate in CIHow a model or prompt change moves through an eval gate in CI: capability and adversarial evals against a versioned suite, a threshold that ships the release or blocks it and files the result against the registry. Generated from the Body of Knowledge.Open interactive diagram (opens in a new tab)

Objectives

Make “give every control teeth” concrete: give a testable property a consequence, so failure stops a release instead of filing a finding.

Target users

AI governance engineer, ML engineer, platform team.

Impacted stakeholders

Model owners, users exposed to the system, auditors.

Relevant principles

Give every control teeth; build the control at the earliest point it can block.

Context

A model or agent that changes (retrained, re-prompted, given a new tool) and a pipeline that already runs tests for functional correctness.

Problem

Evaluations run once before launch and pasted into a slide prove nothing after the next change. A review board that can only rate findings cannot stop a scheduled launch. Without a gate, evaluation is research, not control.

Solution

Version an eval suite alongside the model. Run at least one capability eval and one adversarial eval in CI (for example with Inspect, promptfoo, Garak or Giskard; illustrative). Set a threshold that traces to a named failure mode or obligation. Fail the pipeline below the threshold. Emit a structured result (suite id, model version, score, threshold, pass/fail, timestamp) filed against the registry entry.

Illustrative schema for the result:

{
  "suite_id": "injection-resistance.v4",
  "model_version": "csa-01@2026-09-18",
  "score": 0.982,
  "threshold": 0.95,
  "result": "pass",
  "timestamp": "2026-09-18T14:22:03Z"
}

Consequences

Regressions are caught before production and evidence accrues automatically. The trade-off is eval maintenance, run-time cost in CI, and the need to tune thresholds to avoid flaky gates.

Policy Card; Adversarial Red-Team Suite; Continuous Assurance Telemetry; Machine-Readable Evidence (OSCAL); Model Card as Control Evidence.

Maps to: EU AI Act Art. 15, Art. 55 · ISO/IEC 42001 · NIST AI RMF (Measure) · OWASP Agentic ASI01/ASI02 · Layer 03 Evals & Red Teaming as Evidence.

Threat IDs follow the OWASP Top 10 for Agentic Applications 2026 ¹ and function labels the NIST AI RMF². Mappings are illustrative, not a claim of conformity.

Sources

[1] Top 10 for Agentic Applications 2026 (ASI IDs). OWASP GenAI Security Project. 2025-12-09. https://genai.owasp.org/resource/owasp-top-10-for-agentic-applications-for-2026/ (verified: primary)
[2] AI Risk Management Framework (AI RMF 1.0; Govern, Map, Measure, Manage). NIST. 2023-01-26. https://www.nist.gov/itl/ai-risk-management-framework (verified: primary)

Edit this page on GitHub

Cite this pattern

García Aibar, J. (2026). Pattern: Eval Gate in CI. In AI Governance Engineering: The Thesis & Body of Knowledge (v0.5.0), chapter 05, Patterns. https://doi.org/10.5281/zenodo.22956197. https://aigovernanceengineer.com/patterns/eval-gate-in-ci. CC BY 4.0

BibTeX

@misc{aige2026bok,
  author  = {Jorge García Aibar},
  title   = {{AI Governance Engineering: The Thesis \& Body of Knowledge}},
  chapter = {05. Patterns: Eval Gate in CI},
  year    = {2026},
  version = {0.5.0},
  doi     = {10.5281/zenodo.22956197},
  url     = {https://aigovernanceengineer.com/patterns/eval-gate-in-ci},
  note    = {Version 0.5.0}
}

Obligations this pattern evidences

Cases where it applies

Related controls