Pattern: Runtime Guardrail

Input and output guardrails on the live request path that enforce the system's Policy Card on every call and emit a decision event for each one.

By Jorge García Aibar
v0.5.0
Updated 2026-09-28
CC BY 4.0

Layer 04 · Runtime Controls & Observability In the chapter 05 catalogue

One working control on the engineering side of AI governance.

Summary: Place input and output guardrails on the model or agent’s runtime path that enforce its Policy Card on every live request and emit a decision to telemetry and to the circuit breaker. The guardrail is where a policy written in layer 01 and a threshold tested in layer 03 become an action taken on a real call, not a claim about one.

In short

The Runtime Guardrail is a runtime control that places input and output guardrails on a model or agent’s live request path, enforcing the system’s Policy Card on every call. It solves the gap left by point-in-time policies and evals: once in production, the system meets prompt injection, unsafe outputs and tool calls no one reviewed, and a guardrail that only logs is observability mistaken for control. Use it for any agent or model acting on live inputs whose Policy Card and eval thresholds have no runtime enforcement point. The input guardrail screens prompts and retrieved context for injection and policy-violating requests; the output guardrail screens generations and tool calls for unsafe content, data leakage and out-of-scope actions. Every call emits a structured decision event to the assurance store, and a defined breach signals the circuit breaker. Its illustrative mappings are EU AI Act Art. 14 and 15, ISO/IEC 42001, the NIST AI RMF Manage function and OWASP Agentic ASI02 and ASI03.

Runtime GuardrailInput and output guardrails on the model or agent path enforce the same policy card that CI evaluated and emit a decision event on every call to the assurance store and, on a breach, to the circuit breaker. A guardrail without events is a filter, not a control. Generated from the Body of Knowledge.Open interactive diagram (opens in a new tab)

Objectives

Enforce policy at the point of action, on inputs and outputs no eval anticipated, and make each enforcement a structured event the assurance and incident layers can consume.

Target users

AI governance engineer, ML engineer, security engineer, platform team.

Impacted stakeholders

Users, model owners, affected persons, incident responders, auditors.

Relevant principles

Give every control teeth; instrument the build to produce its own proof.

Context

An agent or model in production, acting on live inputs, whose Policy Card and eval thresholds exist but have no runtime enforcement point, so a rule proven in CI is unguarded the moment the system meets an input no test covered.

Problem

Policies and evals are point-in-time; the system then meets prompt injection, unsafe outputs and tool calls no one reviewed. A guardrail that only logs is observability mistaken for control: the system watches itself fail in high resolution. Without an enforcement point that can block and emit, the runtime is the gap between a tested control and an uncontrolled action.

Solution

Put a guardrail on both sides of the model/agent path. The input guardrail screens prompts and retrieved context for injection and policy-violating requests before they reach the model; the output guardrail screens generations and tool calls for unsafe content, data leakage and out-of-scope actions before they take effect. Enforce the same Policy Card evaluated in CI¹, so the runtime decision and the pipeline decision share one rule. On every call emit a structured event, {agent, direction (input/output), rule_id, decision (allow/block/redact), timestamp}, to the assurance store (Continuous Assurance Telemetry) and, on a defined breach, signal the circuit breaker (Kill Switch / Circuit Breaker). Guardrail frameworks realise this as a category; the OWASP Agent Control Standard names the runtime-control surface². This is distinct from the kill switch: the guardrail decides one call at a time and stays in the request path; the breaker withdraws the agent’s autonomy wholesale when the guardrail’s signals cross a threshold.

Singapore’s framework for agentic AI puts the same control at run time: “Runtime controls address this by monitoring and intervening during execution, such as rate limits to prevent excessive tool use or input validation” (IMDA, section 2.3.1, p. 34)⁵.

Consequences

The tested control holds on live traffic and every enforcement leaves evidence; the guardrail is also the sensor the breaker and the incident pipeline read. The cost is per-call latency, false positives to tune, and keeping the runtime rule in sync with the Policy Card and eval thresholds.

Policy Card; Kill Switch / Circuit Breaker; Agent Identity & Scoped Credentials; Continuous Assurance Telemetry; Human-in-the-loop Gate; Eval Gate in CI.

Maps to: EU AI Act Art. 14, Art. 15 · ISO/IEC 42001 · NIST AI RMF (Manage) · OWASP Agentic ASI02/ASI03 · Layer 04 Runtime Controls & Observability.

Threat IDs follow the OWASP Top 10 for Agentic Applications 2026 ³ and function labels the NIST AI RMF⁴. Mappings are illustrative, not a claim of conformity.

Sources

[1] Policy Cards: machine-readable, deployment-layer governance artefacts for AI agents, linked to enforcement and audit pipelines (arXiv 2510.24383). 2025-10. https://arxiv.org/abs/2510.24383 (verified: primary)
[2] OWASP GenAI Security Project unveils the Agent Control Standard (ACS) and 2026 Top 10 for LLM Applications. OWASP GenAI Security Project. 2026-09-01. https://genai.owasp.org/2026/09/01/owasp-genai-security-project-unveils-2026-top-10-for-llm-applications-new-agent-control-standard-and-sponsors-as-community-tops-30000-members/ (verified: primary)
[3] Top 10 for Agentic Applications 2026 (ASI IDs). OWASP GenAI Security Project. 2025-12-09. https://genai.owasp.org/resource/owasp-top-10-for-agentic-applications-for-2026/ (verified: primary)
[4] AI Risk Management Framework (AI RMF 1.0; Govern, Map, Measure, Manage). NIST. 2023-01-26. https://www.nist.gov/itl/ai-risk-management-framework (verified: primary)
[5] Model AI Governance Framework for Agentic AI, v1.5 (section 2.3.1, p. 34: runtime controls that monitor and intervene during execution; published 2026-05-20, updated 2026-06-05). IMDA. 2026-06-05. https://www.imda.gov.sg/-/media/imda/files/about/emerging-tech-and-research/artificial-intelligence/mgf-for-agentic-ai.pdf (verified: primary)

Edit this page on GitHub

Cite this pattern

García Aibar, J. (2026). Pattern: Runtime Guardrail. In AI Governance Engineering: The Thesis & Body of Knowledge (v0.5.0), chapter 05, Patterns. https://doi.org/10.5281/zenodo.22956197. https://aigovernanceengineer.com/patterns/runtime-guardrail. CC BY 4.0

BibTeX

@misc{aige2026bok,
  author  = {Jorge García Aibar},
  title   = {{AI Governance Engineering: The Thesis \& Body of Knowledge}},
  chapter = {05. Patterns: Runtime Guardrail},
  year    = {2026},
  version = {0.5.0},
  doi     = {10.5281/zenodo.22956197},
  url     = {https://aigovernanceengineer.com/patterns/runtime-guardrail},
  note    = {Version 0.5.0}
}

What Runtime Guardrail connects to

Family

Item

Relation

Related patterns

Agent Identity & Scoped Credentials

Related pattern

Related patterns

Continuous Assurance Telemetry

Related pattern

Related patterns

Eval Gate in CI

Related pattern

Related patterns

Human-in-the-loop Gate

Related pattern

Related patterns

Kill Switch / Circuit Breaker

Related pattern

Related patterns

Policy Card

Related pattern

Obligations

EU AI Act Art. 14: human oversight

Names this pattern

Obligations

EU AI Act Art. 15: accuracy, robustness and cybersecurity

Names this pattern

Obligations

EU AI Act Art. 15(4): feedback loops in systems that continue to learn

Names this pattern

Obligations

NIST AI RMF MANAGE

Names this pattern

Obligations

OWASP Top 10 for Agentic Applications 2026

Names this pattern

Controls

AIGE-CTL-AGENT-035 Agents that watch agents are governed agents

Names this pattern

Controls

AIGE-CTL-AGENT-016 Code runs only in a sandbox

Names this pattern

Controls

AIGE-CTL-AGENT-008 Execution budgets

Names this pattern

Controls

AIGE-CTL-AGENT-023 Memory write gate and rollback

Names this pattern

Controls

AIGE-CTL-EVAL-002 Network Egress Control

Names this pattern

Controls

AIGE-CTL-AGENT-017 Output and egress filter

Names this pattern

Controls

AIGE-CTL-AGENT-007 Runtime guardrail on every tool call

Names this pattern

Controls

AIGE-CTL-AGENT-005 Tool allow-list, deny by default

Names this pattern

Controls

AIGE-CTL-EVAL-004 Tool and Action Mediation

Names this pattern

Cases

Agents in a cyber range with open internet took unsanctioned actions against real people

Names this pattern

Cases

Source code pasted into a public chatbot (reported)

Names this pattern

Cases

Claude models reached real systems from a misconfigured third-party cyber evaluation

Names this pattern

Cases

An agent in training reached a public chatbot through the sandbox DNS resolver

Names this pattern

Cases

The Garante's ChatGPT order: launch before a lawful basis

Names this pattern

Cases

An internally deployed model published a researcher's GitHub token in a public repository

Names this pattern

Cases

Moffatt v. Air Canada: the chatbot's answer is the company's answer

Names this pattern

Cases

NYC MyCity: a government chatbot that advised breaking the law

Names this pattern

Cases

OpenAI agents and Hugging Face: an evaluation environment that was not isolated

Names this pattern

Cases

Agents in training shared a file through a public file-hosting service

Names this pattern

Obligations this pattern evidences

Register rows that list this pattern among the patterns that build their evidence. Mappings are illustrative, not a claim of conformity.

When these obligations apply Each obligation this pattern evidences, on the date it first applies, drawn by its status in the register, and on each later dated step (a square, filled once reached); the dashed line is the date the rows were last reviewed. What comes before the line is already due, so the pattern's evidence is needed now; what comes after it sets the deadline to build it by.

Data table

When the obligations Runtime Guardrail evidences apply
Date	Obligation	Step	Status
2027-12-02	EU AI Act Art. 14: human oversight	Applies from	Deferred
2027-12-02	EU AI Act Art. 15: accuracy, robustness and cybersecurity	Applies from	Deferred
2027-12-02	EU AI Act Art. 15(4): feedback loops in systems that continue to learn	Applies from	Deferred
2028-08-02	EU AI Act Art. 14: human oversight	Later step: Applies to Annex I embedded (product safety-component) systems	Deferred
2028-08-02	EU AI Act Art. 15: accuracy, robustness and cybersecurity	Later step: Applies to Annex I embedded (product safety-component) systems	Deferred
2028-08-02	EU AI Act Art. 15(4): feedback loops in systems that continue to learn	Later step: Applies to Annex I embedded (product safety-component) systems	Deferred
2030-08-02	EU AI Act Art. 14: human oversight	Later step: Deadline for legacy high-risk systems intended for use by public authorities (Art. 111(2))	Deferred
2030-08-02	EU AI Act Art. 15: accuracy, robustness and cybersecurity	Later step: Deadline for legacy high-risk systems intended for use by public authorities (Art. 111(2))	Deferred
2030-08-02	EU AI Act Art. 15(4): feedback loops in systems that continue to learn	Later step: Deadline for legacy high-risk systems intended for use by public authorities (Art. 111(2))	Deferred
No date	NIST AI RMF MANAGE		Voluntary
No date	OWASP Top 10 for Agentic Applications 2026		Voluntary

Obligations this pattern evidences

Cases where it applies

Related controls