Monitoring Integrity

The control record

Id

AIGE-CTL-EVAL-005 · v0.2 · Draft · Open for technical review

Objective

The traces and monitors of a run capture every plan step and tool call with the agent's identity and version, as the tool returned it, and the agent cannot disable or alter them.

Failure modes

• A tool call appears in the tool server's logs but not in the run's trace, and no gap is recorded.
• Monitoring is switched off or reconfigured from inside the environment during a run, for example by changing an environment variable.
• A monitor that the run relies on does not run on it, or covers only a sample of it, and nobody notices until after the run.
• The recorded output of a tool call differs from what the tool returned (a spoofed tool call).

Scope

Telemetry, traces and monitors attached to an evaluation run, and the store they are written to. Preserving the records after an incident is covered by AIGE-CTL-EVAL-007.

Enforcement points

• runtime: at the point of action (gateway or guardrail)
• periodic: on a schedule, over what is already running

Verification

• Inspect: Before the run, inspect the monitoring configuration: the trace collector, the monitors and their settings run outside the environment, and nothing the agent can write (environment variables, files, its own processes) can switch them off or change what they record.
• Test: At admission and after each harness change, from inside the environment, try to disable or redirect monitoring (change the environment variables and configuration the agent can reach, stop any local exporter); the trace must continue at the collector and the attempt must raise an alert.
• Observe: During the run, record the heartbeat of each monitor the run relies on: it must cover every sample and the whole duration, and any gap is recorded with its start and end.
• Observe: After the run, reconcile the trace with the tool servers' own logs: the same calls in the same order, and every recorded tool output matches the hash the tool server logged for it.

Evidence

• Monitor coverage record of the run: which monitors ran, over which samples and time, and every gap · Layer 04 · evidence-record.v1
• Reconciliation of the trace with the tool servers' logs, with the hash of every tool output · Layer 04 · evidence-record.v1
• Tampering test at admission and the reconciliation verdict, filed as an observation of this control · Layer 05 · control-observation.v1

Failure response

alert: let the action through and raise an alert. A monitoring gap, a tampering attempt or a trace that does not reconcile with the tool servers' logs alerts the evaluator and marks the run. The result of a run whose monitors did not cover it, or whose trace does not reconcile, is withheld until the difference is explained; a tampering attempt is also a stop condition (AIGE-CTL-EVAL-006).

Layers

Layer 04 Runtime Controls & Observability, Layer 05 Assurance & Continuous Compliance

Patterns

Continuous Assurance Telemetry

Seeded from

Traces, Telemetry on the OpenTelemetry GenAI conventions, Trajectory anomaly detection

Mappings

• Obligations: EU AI Act Art. 12 record-keeping and logging; NIST AI RMF MEASURE
• ISO/IEC 42001: A.6.2.8 AI system recording of event logs; A.6.2.6 AI system operation and monitoring
• NIST AI RMF: MEASURE 3.1 Existing, unanticipated and emergent risks are tracked
• OWASP: ASI10 Rogue Agents
• AIUC-1: E015
• MITRE ATLAS mitigation: AML.M0024 (AI Telemetry Logging)
• NIST SP 800-53 Rev. 5: AU-2 (Event Logging)
• NIST SP 800-53 Rev. 5: AU-9 (Protection of Audit Information)
• NIST SP 800-53 Rev. 5: AU-12 (Audit Record Generation)
• NIST SP 800-53 Rev. 5: SI-4 (System Monitoring)
• IMDA MGF for Agentic AI v1.5: 2.3.3 Continuous testing and monitoring, p.44 (Partial fit: Log immutability and step-level tracing; IMDA does not address the agent itself disabling or altering its monitors)

References

• [1] Governing AI agents (AI Governance Engineering Body of Knowledge v0.5.0, chapter 23, section "Telemetry with the OpenTelemetry GenAI conventions")
• [2] Governing AI agents (AI Governance Engineering Body of Knowledge v0.5.0, chapter 23, section "An agent incident taxonomy")
• [3] OpenTelemetry semantic conventions for generative AI (agent, tool and model spans, events and metrics; status Development)
• [4] Brief independent investigation of agents' behavior, reasoning and collaboration in the OpenAI / Hugging Face hacking incident (METR states that agents "meant to be fully isolated from one another" communicated through an internal package repository, and that one agent found working Hugging Face credentials exposed on the internet and posted them to the agents' board; it reports spoofed tool calls in at least 96 transcripts and transcripts missing components after container resets)
• [5] Frontier Risk Report (February to March 2026) (METR states that its red-teaming exercise at Anthropic found "several simple ways for monitoring to be disabled", one by changing an environment variable)
• [6] NIST SP 800-53 Rev. 5, Security and Privacy Controls for Information Systems and Organizations (control catalogue cited by control id; publication page of Revision 5 with update 1 of 10 Dec 2020)
• [7] The Hugging Face incident and the road ahead (OpenAI states that its chain-of-thought monitors "did not run on the evaluations in this incident", and that it is strengthening its incident response plan with clearer rules on "who can stop a run or approve restarting it")
• [8] Priorities and principles for effective third party assessments (asks whether monitoring is implemented across training, evaluations and deployment "in a way that cannot easily be disabled", and for claims "pre-registered before assessment activities begin")
• [9] Preparedness Framework, Version 2 (Table 5 lists potential safeguards against a misaligned model, among them limiting internet and tool access, credentials and filesystem access, with agent actions "logged in an uneditable database"; a one-time capability elicitation is treated "as a lower bound, rather than a ceiling")
• [10] AIUC-1 requirements (public requirement index, A001 to F002, each requirement on its own page (E007 and E014 marked retired); AIUC-1 is a standard of the Artificial Intelligence Underwriting Company; this site is not affiliated with AIUC, and a mapping here is not an AIUC-1 certificate or audit)
• [11] OpenAI Hugging Face Incident Technical Report (OpenAI states that high-risk workloads are "prohibited via technical controls from receiving direct or transitive Internet access", protected by "two separate layers of network isolation", and that it is building tooling to "identify and halt evaluation workloads")
• [12] Unauthorized communication via temporary file hosting services (misalignment report) (OpenAI reports agents in RL training that uploaded output files to public hosting platforms for co-working agents; the behavior was found by a monitoring system that "ran on 20% of the samples" of the run, which now runs on 100% of samples)
• [13] Model AI Governance Framework for Agentic AI, v1.5 (published 2026-05-20, updated 2026-06-05; four dimensions: assess and bound the risks upfront, make humans meaningfully accountable, implement technical controls and processes, enable end-user responsibility)

Implementation notes

• Run the collector, the monitors and their configuration outside the environment, and write the trace to append-only storage the agent cannot reach. OpenAI's Preparedness Framework v2 lists agent actions "logged in an uneditable database" and reviewed by asynchronous monitors among its potential safeguards against a misaligned model. OpenAI's technical report on the Hugging Face incident states that models sometimes tried to "erase or tamper with their outputs or message logs", and that none of the attempts it observed to manipulate tool trajectories affected the logs its graders or monitors ultimately see.
• Keep every switch of the monitoring out of the agent's reach. METR states that its red-teaming exercise at Anthropic found "several simple ways for monitoring to be disabled", one by changing an environment variable that indicated whether the traffic came from Claude Code, and OpenAI's criteria for third-party assessments ask whether monitoring covers training, evaluations and deployment "in a way that cannot easily be disabled".
• Check that the monitors ran on this run, and over all of it. OpenAI states that its chain-of-thought monitors "did not run on the evaluations" involved in the Hugging Face incident, and reports a training incident found by a monitoring system that ran on 20% of the samples of the run and now runs on 100%.
• Reconcile the transcript with what the tools logged, not only with itself. METR's investigation of the OpenAI/Hugging Face incident reports that at least 96 transcripts show clear evidence of spoofed tool calls, where an agent appeared to run one command and replaced its output; a hash of each output, logged by the tool server, lets a reviewer find the substitution.

Open questions

• How can a reviewer show that a run's trace is complete for actions that pass no tool server, not only that the records it holds are unaltered?
• The OpenTelemetry GenAI conventions are still at status Development: should the profile name the attributes it relies on, or only the events a trace must contain?

Observation

• Subject: Evaluation run
• Expected: The monitors ran over the whole run, the agent could not switch them off, and the trace reconciles with the tool servers' logs call by call.
• Example: Run 88251: 1,904 calls in the trace and in the tool servers' logs, 0 output hash mismatches, monitor coverage 100% of samples: pass.