AI Governance Engineer
GitHub

An AI governance framework in five layers.

The reference architecture of AI governance engineering: five layers that turn governance from a stack of parallel documents into one system with a data path from policy to proof.

Chapter 04 · Body of Knowledge

What is an AI governance framework?

An AI governance framework is the set of rules, roles, controls and evidence that keeps an organisation's AI systems within bounds. Here it is built as five layers, read as a build order: Govern-as-Code writes the rules, Inventory & Transparency records what is running, Evals & Red Teaming as Evidence tests it, Runtime Controls & Observability enforces the rules on the live call, and Assurance & Continuous Compliance turns the evidence into proof.

Most frameworks list the same components in different words: an inventory of AI systems and a way to classify them by risk, policies written so they can be checked, impact assessments, tests before release, human oversight where it is needed, runtime controls, an incident process and the evidence that ties each control to the obligation it serves. The five layers below produce each of them. For what AI governance is and how the EU AI Act, ISO/IEC 42001 and the NIST AI RMF compare, read AI governance, explained; for every chapter behind this stack, open the full body of knowledge.

Follow one artefact from policy to proof.

Evidence flows up 05 Assurance & Continuous Compliance 04 Runtime Controls & Observability 03 Evals & Red Teaming as Evidence 02 Inventory & Transparency 01 Govern-as-Code

Layer 01 · Govern-as-Code

A policy verdict tied to a commit, a pull request or a deploy, machine-readable and reproducible.

Artefacts

  • Policy-as-code library under version control
  • Crosswalks that map each policy to the frameworks it serves
  • CI/CD wiring that runs the policy at the right gate
  • A structured policy verdict (allow/deny, rule id, input hash, timestamp) per evaluation

Maps to

  • EU AI Act Art. 9 (risk management)
  • ISO/IEC 42001
  • NIST AI RMF (Govern)
  • CSA AICM
  • OWASP Agentic ASI02/ASI03
Read layer 01 in chapter 04
Evidence flows up: each layer emits a structured artefact that layer 05 aggregates into audit-ready assurance.

What fills each layer.

The site's registers, counted layer by layer: the obligations an artefact answers, the patterns and controls that build it, the tools that run it, the learning-path nodes that teach it and the role's workflows that own it.

Layer Obligations Patterns Controls Tools Path nodes Workflows
01 Govern-as-Code 57 6 4 11 6 2
02 Inventory & Transparency 81 9 21 24 5 3
03 Evals & Red Teaming as Evidence 54 3 8 28 5 2
04 Runtime Controls & Observability 85 10 45 26 6 2
05 Assurance & Continuous Compliance 92 5 15 13 3 2
Records per stack layer 182 obligations, 33 patterns, 93 controls, 96 tools, 37 path nodes and 7 workflows; each bar is a share of the largest count in its column. Patterns and controls count in their home layer; obligations and tools count in every layer they name, the workflow that spans all five counts in each, and the 12 cross-cutting path nodes count in none. A number links to that layer's list where one exists.

A build order, from policy to proof.

The three questions Three panels (what AI is running, what it is allowed to do, and what evidence proves it), each with the stack layers that answer it. The spine of the Body of Knowledge What AI is running? Layer 02 Inventory & Transparency What is it allowed to do? Layer 01 Govern-as-Code Layer 04 Runtime Controls & Observability What evidence proves it? Layer 03 Evals & Red Teaming as Evidence Layer 05 Assurance & Continuous Compliance
The three questions The three questions every AI governance function must answer from live systems, and the stack layers that answer each. If you cannot answer one of them for a system today, that is your first task. Drawn from chapter 01. Permalink, downloads and citation
Text description

Three panels, one per question. What AI is running? Answered by Layer 02 Inventory & Transparency: the inventory and the agent registry, fed by a runtime data path. What is it allowed to do? Answered by Layer 01 Govern-as-Code and Layer 04 Runtime Controls & Observability: identity before autonomy, scope before action. What evidence proves it? Answered by Layer 03 Evals & Red Teaming as Evidence and Layer 05 Assurance & Continuous Compliance: evidence as a by-product of the build.

  1. A build order, not an org chart Each layer produces an artefact the layer above consumes: Policy → Inventory → Evals → Runtime → Assurance.
  2. Evidence flows up A policy verdict, a registry entry, an eval result and a guardrail decision are structured artefacts that layer 05 aggregates into audit-ready evidence.
  3. It answers the three questions Layer 02 answers what AI is running; layers 01 and 04 answer what it is allowed to do, one writing the bound and one enforcing it on the live call; layers 03 and 05 answer what evidence proves it.

Read in chapter 04

AI governance and the frameworks this stack answers

Policy, inventory, evals, runtime, assurance.

01Govern-as-Code02Inventory & Transparency03Evals & Red Teaming as Evidence04Runtime Controls & Observability05Assurance & Continuous Compliance

Govern-as-Code

Write the rule as code

Established
Question it answers
What is it allowed to do?
What it proves
A policy verdict tied to a commit, a pull request or a deploy, machine-readable and reproducible.
Artefacts
  • Policy-as-code library under version control
  • Crosswalks that map each policy to the frameworks it serves
  • CI/CD wiring that runs the policy at the right gate
  • A structured policy verdict (allow/deny, rule id, input hash, timestamp) per evaluation
Tool categories
  • Policy engines
    • OPA/Rego
    • Cedar
    • OPAL
  • Policy testing and authoring
    • Conftest
    • The Rego Playground
  • Admission control
    • OPA Gatekeeper
    • Kyverno
  • Machine-readable policy artefacts (proposed)
    • Policy Cards
    • OWASP Agent Control Standard
Maps to
  • EU AI Act Art. 9 (risk management)
  • ISO/IEC 42001
  • NIST AI RMF (Govern)
  • CSA AICM
  • OWASP Agentic ASI02/ASI03
Controls anchored here
4 draft reference controls in the open control profiles, each with the evidence it must leave.
Read layer 01 in chapter 04

Inventory & Transparency

Inventory what is running

Established
Question it answers
What AI is running?
What it proves
A registry entry and its attached documents, ideally written by a deployment pipeline rather than typed by hand.
Artefacts
  • Agent registry: the runtime-aware inventory of every model, service and agent, each with an owner, a scope and a status
  • Transparency documents: model cards and data cards
  • AIBOM, the bill of materials for an AI system
  • FRIA and DPIA references linked to the registry entry
Tool categories
  • Registries and governance suites
    • ServiceNow
    • Credo AI
  • Agent-discovery tools
    • Zenity
  • AIBOM formats and generators
    • CycloneDX ML-BOM
    • SPDX 3.0 AI profile
    • OWASP AIBOM generator
    • AIsbom
  • Model and data card tooling
    • Hugging Face model cards
    • Croissant (ML dataset metadata)
  • FRIA/DPIA tooling
    • CNIL PIA
    • Canada Algorithmic Impact Assessment
  • PII detection and redaction
    • Presidio
  • Data and experiment versioning
    • DVC
    • MLflow
    • lakeFS
    • OpenLineage
  • Model signing and artefact scanning
    • Sigstore (cosign)
    • OpenSSF model signing (model-transparency)
    • in-toto
    • SLSA
    • ModelScan
    • picklescan
Maps to
  • EU AI Act Art. 11 (technical documentation)
  • EU AI Act Art. 49/71 (registration and the EU database)
  • EU AI Act Art. 50 (transparency)
  • ISO/IEC 42001
  • NIST AI RMF (Map)
  • CSA AICM
  • OWASP Agentic ASI10
Controls anchored here
21 draft reference controls in the open control profiles, each with the evidence it must leave.
Read layer 02 in chapter 04

Evals & Red Teaming as Evidence

Run evals as evidence

New
Question it answers
What evidence proves it?
What it proves
A structured eval result: pass or fail against a threshold, versioned alongside the model it tested.
Artefacts
  • Eval suites and the eval gate that runs them
  • Capability and quality evals (groundedness, regression against a golden set)
  • Adversarial and red-team evals (jailbreaks, prompt injection, tool misuse)
  • Safety-threshold evals tied to a policy from layer 01
Tool categories
  • Evaluation frameworks
    • Inspect
    • promptfoo
    • DeepEval
    • HELM
  • Adversarial and vulnerability probes
    • Garak
    • Mindgard
    • Giskard
    • PyRIT
  • Safety and red-team benchmarks
    • MLCommons AILuminate
    • HarmBench
    • JailbreakBench
  • Retrieval-augmented quality
    • Ragas
  • Data validation and quality
    • Great Expectations
    • Deequ
    • pandera
    • TensorFlow Data Validation
    • Deepchecks
  • Fairness toolkits
    • Fairlearn
    • AI Fairness 360
    • Aequitas
  • Explainability libraries
    • SHAP
    • LIME
    • Captum
    • InterpretML
    • DiCE
  • Practice grounds
    • Gandalf (Lakera)
Maps to
  • EU AI Act Art. 15 (accuracy, robustness, cybersecurity)
  • EU AI Act Art. 55 (GPAI systemic-risk evaluation)
  • ISO/IEC 42001
  • NIST AI RMF (Measure)
  • CSA AICM
  • OWASP Agentic ASI01/ASI02
Controls anchored here
8 draft reference controls in the open control profiles, each with the evidence it must leave.
Read layer 03 in chapter 04

Runtime Controls & Observability

Hold the line at runtime

New
Question it answers
What is it allowed to do?
What it proves
A stream of runtime decisions and traces.
Artefacts
  • Guardrails: input/output filters and tool-call mediation at the enforcement point
  • Observability: tracing and monitoring that turn agent behaviour into a control signal
  • Agent runtime identity: a workload identity, a bounded scope and a tested kill switch per actor
Tool categories
  • Guardrail frameworks
    • NVIDIA NeMo Guardrails
    • Meta LlamaFirewall
    • Lakera
    • Guardrails AI
    • Llama Guard
  • Observability
    • Langfuse
    • Arize Phoenix
    • OpenTelemetry (GenAI semantic conventions)
  • ML and LLM monitoring and drift
    • Evidently
    • NannyML
    • Alibi Detect
  • MCP / tool-call security
    • MCP Inspector
    • Snyk Agent Scan
    • mcp-context-protector
  • Progressive delivery and feature flags
    • Argo Rollouts
    • Flagger
    • OpenFeature
  • Kill switch / circuit breaker
    • Unleash
    • Envoy
    • Resilience4j
  • Agent workload identity
    • SPIFFE/SPIRE
    • Microsoft Entra Agent ID
    • Okta Agent SSO
Maps to
  • EU AI Act Art. 14 (human oversight)
  • EU AI Act Art. 15 (robustness, cybersecurity)
  • EU AI Act Art. 12 (logging)
  • ISO/IEC 42001
  • NIST AI RMF (Manage)
  • CSA AICM
  • OWASP Agentic ASI02/ASI03/ASI10
Controls anchored here
45 draft reference controls in the open control profiles, each with the evidence it must leave.
Read layer 04 in chapter 04

Assurance & Continuous Compliance

Close with continuous assurance

Established
Question it answers
What evidence proves it?
What it proves
A live assurance store that any of the lower layers writes into and an auditor can read from.
Artefacts
  • Machine-readable evidence in a standard format (OSCAL)
  • Framework mappings generated from the evidence, not maintained beside it
  • Incident and reporting plumbing that meets obligations on the clock
Tool categories
  • Evidence format
    • OSCAL
    • CSA AICM OSCAL bundle
  • OSCAL tooling (open source)
    • compliance-trestle
    • NIST oscal-cli
    • Lula
  • GRC and AI-governance suites
    • Vanta
    • Drata
    • OneTrust
    • watsonx.governance
    • Holistic AI
    • Saidot
Maps to
  • EU AI Act Art. 17 (quality management)
  • EU AI Act Art. 72 (post-market monitoring)
  • EU AI Act Art. 73 (serious incident reporting)
  • ISO/IEC 42001, ISO/IEC 42005
  • NIST AI RMF (Govern, Manage)
  • CSA AICM
Controls anchored here
15 draft reference controls in the open control profiles, each with the evidence it must leave.
Read layer 05 in chapter 04

Three layers established, two the AI forces.

Established engineering practice

Govern-as-Code (01), Inventory & Transparency (02) and Assurance & Continuous Compliance (05) come almost unchanged from GRC engineering: policy as code, the asset inventory and machine-readable evidence, a specialisation of a method that already works.

New, forced by AI

Evals and red-teaming as controls (03) and agent identity and runtime control (04) are what AI forces the discipline to add: a model whose behaviour must be tested and an autonomous actor acting under delegated authority have no analogue in classic GRC.

The minimum viable stack, for a team of one.

A team of one cannot build all five layers at depth, but it can build the spine thinly, end to end: one vertical slice that touches every layer beats one layer built out and four left on paper.

The minimum viable stack Five ordered steps (see it, rule it, test it, contain it, prove it), one per stack layer, chained top to bottom. One vertical slice, every layer 1 See it Layer 02 Inventory & Transparency A registry a deploy writes to 2 Rule it Layer 01 Govern-as-Code One policy with teeth 3 Test it Layer 03 Evals & Red Teaming as Evidence One adversarial eval gate 4 Contain it Layer 04 Runtime Controls & Observability Identity and a tested kill switch 5 Prove it Layer 05 Assurance & Continuous Compliance Evidence as a by-product
The minimum viable stack The five ordered steps that stand the whole stack up thinly, one per layer, for a team of one. Do step one this week. Drawn from chapter 04. Permalink, downloads and citation
Text description

Five ordered steps for a team of one, in the order see it, rule it, test it, contain it, prove it. Step 1, Layer 02: a registry a deploy writes to, with an owner and a scope per entry. Step 2, Layer 01: one policy with teeth, as code, in the pipeline, blocking on failure. Step 3, Layer 03: one adversarial eval against your highest-risk agent, wired so a regression fails the build. Step 4, Layer 04: every agent under its own identity with a scope, and a tested way to stop it. Step 5, Layer 05: each of the above emits a structured, timestamped record into one store.

  1. Layer 02 first, minimally A registry that a deploy writes to, with an owner and a scope per entry, so you can answer "what is running and who owns it?" from a live source.
  2. One policy in layer 01 with teeth A single rule that matters (no deploy without a registered owner, or a data-residency check) as code, in the pipeline, blocking on failure.
  3. One eval gate in layer 03 One adversarial eval against your highest-risk agent, wired so a regression fails the build; reuse an open framework rather than writing your own harness.
  4. Identity and a kill switch in layer 04 Every agent under its own identity with a scope, and a tested way to stop it: the cheapest control with the largest blast-radius reduction.
  5. Evidence as a by-product in layer 05 Have each of the above emit a structured, timestamped record into one store; not an OSCAL pipeline yet, just a refusal to rely on screenshots.

Read in chapter 04

Tool categories are the substance; brands are illustrative.

Category Tool Layer
Policy engines OPA/Rego 01
Cedar
OPAL
Policy testing and authoring Conftest 01
The Rego Playground
Admission control OPA Gatekeeper 01
Kyverno
Machine-readable policy artefacts (proposed) Policy Cards 01
OWASP Agent Control Standard
Registries and governance suites ServiceNow 02
Credo AI
Agent-discovery tools Zenity 02
AIBOM formats and generators CycloneDX ML-BOM 02
SPDX 3.0 AI profile
OWASP AIBOM generator
AIsbom
Model and data card tooling Hugging Face model cards 02
Croissant (ML dataset metadata)
FRIA/DPIA tooling CNIL PIA 02
Canada Algorithmic Impact Assessment
PII detection and redaction Presidio 02
Data and experiment versioning DVC 02
MLflow
lakeFS
OpenLineage
Model signing and artefact scanning Sigstore (cosign) 02
OpenSSF model signing (model-transparency)
in-toto
SLSA
ModelScan
picklescan
Evaluation frameworks Inspect 03
promptfoo
DeepEval
HELM
Adversarial and vulnerability probes Garak 03
Mindgard
Giskard
PyRIT
Safety and red-team benchmarks MLCommons AILuminate 03
HarmBench
JailbreakBench
Retrieval-augmented quality Ragas 03
Data validation and quality Great Expectations 03
Deequ
pandera
TensorFlow Data Validation
Deepchecks
Fairness toolkits Fairlearn 03
AI Fairness 360
Aequitas
Explainability libraries SHAP 03
LIME
Captum
InterpretML
DiCE
Practice grounds Gandalf (Lakera) 03
Guardrail frameworks NVIDIA NeMo Guardrails 04
Meta LlamaFirewall
Lakera
Guardrails AI
Llama Guard
Observability Langfuse 04
Arize Phoenix
OpenTelemetry (GenAI semantic conventions)
ML and LLM monitoring and drift Evidently 04
NannyML
Alibi Detect
MCP / tool-call security MCP Inspector 04
Snyk Agent Scan
mcp-context-protector
Progressive delivery and feature flags Argo Rollouts 04
Flagger
OpenFeature
Kill switch / circuit breaker Unleash 04
Envoy
Resilience4j
Agent workload identity SPIFFE/SPIRE 04
Microsoft Entra Agent ID
Okta Agent SSO
Evidence format OSCAL 05
CSA AICM OSCAL bundle
OSCAL tooling (open source) compliance-trestle 05
NIST oscal-cli
Lula
GRC and AI-governance suites Vanta 05
Drata
OneTrust
watsonx.governance
Holistic AI
Saidot

See the patterns, one page each

The stack across a system's life, from development to incidents

Read the stack in full.

The full reference architecture, layer by layer, with the mappings and the evidence; the lifecycle chapters then run it from use case to retirement.

Read chapter 04 in full Open the Body of Knowledge