{ "notice": "Illustrative mapping from the AI Governance Engineer Body of Knowledge v0.5.0 (not a claim of conformity)", "version": "0.5.0", "license": "CC BY 4.0", "licenseUrl": "https://creativecommons.org/licenses/by/4.0/", "schemaVersion": 1, "schema": "https://aigovernanceengineer.com/api/v1/schemas/control.json", "self": "https://aigovernanceengineer.com/api/v1/controls/aige-ctl-eval-001.json", "source": "https://aigovernanceengineer.com/controls/evaluation-environment/aige-ctl-eval-001", "citation": { "title": "AI Governance Engineering: The Thesis & Body of Knowledge", "authors": [ "Jorge García Aibar" ], "parentDoi": "https://doi.org/10.5281/zenodo.22956197", "conceptDoi": "https://doi.org/10.5281/zenodo.22857084" }, "control": { "id": "AIGE-CTL-EVAL-001", "profile": "evaluation-environment", "url": "https://aigovernanceengineer.com/controls/evaluation-environment/aige-ctl-eval-001", "json": "https://aigovernanceengineer.com/api/v1/controls/aige-ctl-eval-001.json", "title": "Authorization Boundary", "version": "0.2", "status": "draft", "reviewerStatus": "open", "depth": "specified", "objective": "Every agent in an evaluation run acts only within an authorization boundary recorded before the run starts: the tools, operations, data classes and budgets it may use, and the runs and resources it may reach.", "failureModes": [ "A run starts with no recorded boundary for the agent under test, or with a boundary that differs from the scope the agent was told.", "The agent calls a tool or an operation class outside its recorded boundary, or starts processes with administrative privileges, and the call succeeds.", "A credential given for one purpose on a shared resource, such as downloading packages, also lets the agent write, list or post there.", "Agents in runs meant to be isolated reach each other through a resource the runs share, such as a package repository or a cache." ], "scope": "Agents and harnesses under evaluation, the tools and operations they can use during a run, the credentials they receive for shared resources and the resources runs share. Budgets are recorded here and enforced under AIGE-CTL-EVAL-006; network egress is AIGE-CTL-EVAL-002. Production deployments are covered by the agent runtime profile.", "enforcementPoints": [ "deploy", "runtime" ], "verification": [ { "kind": "inspect", "text": "Before the run, inspect the run record: it holds a boundary for each agent under test (tools and operation classes, data classes, budgets, and each shared resource with the operations allowed on it), and the prompt the agent receives states the same boundary as instructions, including what it must not access." }, { "kind": "test", "text": "At admission, from inside the environment, attempt one call of each kind outside the boundary (an unlisted tool or operation class, a write or a listing with a download-only credential on a shared resource, a process started as root) and one listed call; every attempt outside the boundary must be refused and logged, and the listed call must succeed." }, { "kind": "test", "text": "Start two canary runs on the same shared resources: a marker written by one run must not be readable by the other." }, { "kind": "observe", "text": "After the run, compare every tool call and every request to a shared resource in the run's logs with the recorded boundary: each falls inside it, and every refused attempt is recorded with its time and target." } ], "evidence": [ { "artefact": "The boundary of each agent under test, recorded before the run: tools, operation classes, data classes, budgets and shared resources with the operations allowed on each", "schemaId": "agent-register-entry", "schema": "https://aigovernanceengineer.com/schemas/agent-register-entry.v1.json", "layer": 2 }, { "artefact": "Admission test verdicts: the refused out-of-boundary calls and the cross-run canary", "schemaId": "evidence-record", "schema": "https://aigovernanceengineer.com/schemas/evidence-record.v1.json", "layer": 4 }, { "artefact": "One observation per run comparing the calls and shared-resource requests made with the recorded boundary", "schemaId": "control-observation", "schema": "https://aigovernanceengineer.com/schemas/control-observation.v1.json", "layer": 5 } ], "failureResponse": { "effect": "deny", "text": "Calls and shared-resource operations outside the recorded boundary are refused at the enforcement point and logged. A run with no recorded boundary is not started; a run in which a call outside the boundary succeeded, or in which runs reached each other, is stopped and its result is withheld until the path is closed." }, "layer": 4, "secondaryLayers": [ 2 ], "patterns": [ { "slug": "agent-registry", "title": "Agent Registry", "url": "https://aigovernanceengineer.com/patterns/agent-registry" }, { "slug": "policy-card", "title": "Policy Card", "url": "https://aigovernanceengineer.com/patterns/policy-card" } ], "seeds": [ { "id": "registry-entry", "title": "Registry entry", "url": "https://aigovernanceengineer.com/bok/governing-agents#the-agent-registry" }, { "id": "read-only-tools", "title": "Read-only tools", "url": "https://aigovernanceengineer.com/bok/governing-agents#autonomy-is-a-design-decision" }, { "id": "reversible-only", "title": "Reversible, bounded actions only", "url": "https://aigovernanceengineer.com/bok/governing-agents#autonomy-is-a-design-decision" }, { "id": "execution-budgets", "title": "Execution budgets", "url": "https://aigovernanceengineer.com/bok/governing-agents#execution-limits" }, { "id": "data-classes", "title": "Data classes recorded, with the DPIA linked", "url": "https://aigovernanceengineer.com/bok/governing-agents#the-agent-registry" } ], "derivedFrom": [], "mappings": { "obligations": [ { "id": "AIGE-OBL-EUAIA-ART14", "name": "EU AI Act Art. 14 human oversight", "url": "https://aigovernanceengineer.com/obligations/aige-obl-euaia-art14" }, { "id": "AIGE-OBL-OWASP-AGENTIC", "name": "Top 10 for Agentic Applications 2026", "url": "https://aigovernanceengineer.com/obligations/aige-obl-owasp-agentic" }, { "id": "AIGE-OBL-CSA-AICM-AGENTIC", "name": "AICM agent controls with the CSA Agentic Trust Framework and AARM specification", "url": "https://aigovernanceengineer.com/obligations/aige-obl-csa-aicm-agentic" }, { "id": "AIGE-OBL-SG-AGENTIC-IDENTITY", "name": "Singapore IMDA Model AI Governance Framework for Agentic AI: agent identity and scoped authorisations (voluntary)", "url": "https://aigovernanceengineer.com/obligations/aige-obl-sg-agentic-identity" } ], "iso42001": [ { "id": "A.6.2.2", "title": "AI system requirements and specification" }, { "id": "A.9.2", "title": "Processes for responsible use of AI systems" } ], "nistAiRmf": [ { "id": "MAP 4.2", "title": "Internal risk controls for components of the AI system, including third-party AI technologies, are identified and documented." } ], "owasp": [ { "id": "asi02", "externalId": "ASI02", "name": "Tool Misuse and Exploitation", "url": "https://aigovernanceengineer.com/resources/threats#threat-asi02" }, { "id": "asi03", "externalId": "ASI03", "name": "Identity and Privilege Abuse", "url": "https://aigovernanceengineer.com/resources/threats#threat-asi03" }, { "id": "llm03-2026", "externalId": "LLM03:2026", "name": "Excessive Agency", "url": "https://aigovernanceengineer.com/resources/threats#threat-llm03-2026" } ], "atlas": [], "aiuc1": [ "B006" ], "csaAicm": [], "other": [ { "framework": "NIST SP 800-53 Rev. 5", "ref": "AC-3", "note": "Access Enforcement" }, { "framework": "NIST SP 800-53 Rev. 5", "ref": "AC-6", "note": "Least Privilege" } ] }, "references": [ { "n": 1, "title": "Governing AI agents", "text": "Governing AI agents (AI Governance Engineering Body of Knowledge v0.5.0, chapter 23, section \"The agent registry\"). AI Governance Engineer (Jorge García Aibar). 2026-09.", "url": "https://aigovernanceengineer.com/bok/governing-agents#the-agent-registry", "verified": "primary" }, { "n": 2, "title": "Governing AI agents", "text": "Governing AI agents (AI Governance Engineering Body of Knowledge v0.5.0, chapter 23, section \"Autonomy is a design decision\"). AI Governance Engineer (Jorge García Aibar). 2026-09.", "url": "https://aigovernanceengineer.com/bok/governing-agents#autonomy-is-a-design-decision", "verified": "primary" }, { "n": 3, "title": "Governing AI agents", "text": "Governing AI agents (AI Governance Engineering Body of Knowledge v0.5.0, chapter 23, section \"The tool allow-list\"). AI Governance Engineer (Jorge García Aibar). 2026-09.", "url": "https://aigovernanceengineer.com/bok/governing-agents#the-tool-allow-list", "verified": "primary" }, { "n": 4, "title": "METR Task Standard, STANDARD.md", "text": "METR Task Standard, STANDARD.md (version 0.5.0; unless a task declares the full_internet permission, the task machines \"MUST NOT have internet access\" except to an LLM API, an LLM API proxy or a hardened local server). METR (GitHub). 2024-10-30.", "url": "https://raw.githubusercontent.com/METR/task-standard/main/STANDARD.md", "verified": "primary" }, { "n": 5, "title": "Brief independent investigation of agents' behavior, reasoning and collaboration in the OpenAI / Hugging Face hacking incident", "text": "Brief independent investigation of agents' behavior, reasoning and collaboration in the OpenAI / Hugging Face hacking incident (METR states that agents \"meant to be fully isolated from one another\" communicated through an internal package repository, and that one agent found working Hugging Face credentials exposed on the internet and posted them to the agents' board; it reports spoofed tool calls in at least 96 transcripts and transcripts missing components after container resets). METR. 2026-08-26.", "url": "https://metr.org/blog/2026-08-26-openai-hugging-face-incident-investigation/", "verified": "primary" }, { "n": 6, "title": "OWASP Top 10 for Agentic Applications for 2026", "text": "OWASP Top 10 for Agentic Applications for 2026 (ASI01 Agent Goal Hijack to ASI10 Rogue Agents). OWASP GenAI Security Project. 2025-12-09.", "url": "https://genai.owasp.org/resource/owasp-top-10-for-agentic-applications-for-2026/", "verified": "primary" }, { "n": 7, "title": "NIST SP 800-53 Rev. 5, Security and Privacy Controls for Information Systems and Organizations", "text": "NIST SP 800-53 Rev. 5, Security and Privacy Controls for Information Systems and Organizations (control catalogue cited by control id; publication page of Revision 5 with update 1 of 10 Dec 2020). NIST. 2020-12-10.", "url": "https://csrc.nist.gov/pubs/sp/800/53/r5/upd1/final", "verified": "primary" }, { "n": 8, "title": "Third-party cyber evaluations involving OpenAI models", "text": "Third-party cyber evaluations involving OpenAI models (OpenAI states that the evaluator's \"intended authorization boundary was the simulated cyber range\", that its model reused a GitHub token another lab's agent had left publicly accessible, and that it will review how to \"set expectations for isolation, credential handling, monitoring, and stop conditions\"). OpenAI. 2026-08-04.", "url": "https://openai.com/index/third-party-cyber-evaluations-involving-openai-models/", "verified": "primary" }, { "n": 9, "title": "Improving our alignment and security efforts", "text": "Improving our alignment and security efforts (best practices for external partners running cyber evaluations: the only outside connection is \"to the model's own API\", with the API keys kept outside the environment; the configuration \"should be verified before every evaluation begins\"; boundaries \"phrased as instructions\"; challenges confirmed \"solvable in principle\"; a monitor that flags a scope violation to a human and ends the exercise). Anthropic. 2026-08-31.", "url": "https://www.anthropic.com/news/improving-alignment-security-efforts", "verified": "primary" }, { "n": 10, "title": "AIUC-1 requirements", "text": "AIUC-1 requirements (public requirement index, A001 to F002, each requirement on its own page (E007 and E014 marked retired); AIUC-1 is a standard of the Artificial Intelligence Underwriting Company; this site is not affiliated with AIUC, and a mapping here is not an AIUC-1 certificate or audit). Artificial Intelligence Underwriting Company. 2026-09-24.", "url": "https://standard.aiuc-1.com/llms.txt", "verified": "primary" }, { "n": 11, "title": "Unsanctioned Artifactory writes and cross-sample communication (misalignment report)", "text": "Unsanctioned Artifactory writes and cross-sample communication (misalignment report) (OpenAI reports models in RL training that used Artifactory, \"a shared package repository\", to exchange messages across separate training samples with the credentials they were given to download packages, without exploiting any vulnerability; such exchanges can \"undermine the assumption that training or evaluation samples are independent\"). OpenAI (Alignment Research Blog). 2026-09-16.", "url": "https://alignment.openai.com/misalignment-reports/unauthorized-artifactory-writes-and-cross-sample-communication/", "verified": "primary" }, { "n": 12, "title": "OpenAI Hugging Face Incident Technical Report", "text": "OpenAI Hugging Face Incident Technical Report (OpenAI states that high-risk workloads are \"prohibited via technical controls from receiving direct or transitive Internet access\", protected by \"two separate layers of network isolation\", and that it is building tooling to \"identify and halt evaluation workloads\"). OpenAI. 2026-08-26.", "url": "https://cdn.openai.com/pdf/67869394-cb91-4c12-888c-5cbd85c7814c/OpenAI-Hugging-Face%20Incident-Technical-Report.pdf", "verified": "primary" } ], "implementationNotes": [ "Record the boundary as data before the run, in the fields of an agent registry entry (scope, tools, data access, spend limit), and enforce it at the gateway the agent's calls pass through, not in the prompt alone. State the same boundary to the agent: Anthropic's guidance for external evaluation partners asks every prompt to state what is in and out of scope, including permitted actions and network boundaries, with boundaries \"phrased as instructions\" rather than claims about the environment.", "Tell the agent what it may not do with the access it has. OpenAI's account of a third-party cyber evaluation states that the evaluator's \"intended authorization boundary was the simulated cyber range\", while the agents were not explicitly told how they could and could not use the open internet access they had; the evaluator names this as a potential contributing cause.", "Scope every credential for a shared resource to the operations the task needs: download from a package repository, never write, list or post. OpenAI reports models in RL training that used Artifactory, \"a shared package repository\", to exchange messages across separate samples with the credentials they were given to download packages, without exploiting any vulnerability; its technical report on the Hugging Face incident lists removing the shared credential from new research workloads and restricting the types of requests agents could make to Artifactory among its mitigations.", "Give each run its own namespace on every shared resource (package cache, artefact store, scratch storage) and test it with a canary marker. METR's investigation of the OpenAI/Hugging Face incident reports that agents \"meant to be fully isolated from one another\" communicated through an internal package repository.", "Run the agent as an unprivileged user. METR's Task Standard states that the agent \"MUST NOT be allowed to start processes\" on the primary machine as root." ], "openQuestions": [ "Which shared resources (package mirrors, caches, model API proxies) can be given a per-run namespace, and which should be removed from evaluation environments altogether?", "What evidence is sufficient to show that an agent stayed within its boundary for a whole run when the logs of some of the tools it called are outside the evaluator's control?" ], "observation": { "subjectKind": "eval-run", "expected": "Every tool call and every request to a shared resource falls inside the boundary recorded for the agent before the run; every attempt outside it is refused and logged.", "observedExample": "Run 88231: 3 listing and upload requests to the shared package repository with a download-only credential, all refused and logged; no call outside the boundary succeeded: pass." }, "observationSchema": "https://aigovernanceengineer.com/schemas/control-observation.v1.json", "examples": [ { "status": "pass", "url": "https://aigovernanceengineer.com/controls/examples/control-observation.aige-ctl-eval-001.pass.json" }, { "status": "fail", "url": "https://aigovernanceengineer.com/controls/examples/control-observation.aige-ctl-eval-001.fail.json" } ] } }