15. Governing deployment and use

New to the field? Start with the definition of AI governance.

Most organisations deploy far more AI than they build. A provider’s design duties are largely discharged when the system is placed on the market; a deployer’s duties begin when the system is put to use, and they run for as long as it runs: use it as instructed, staff its oversight, monitor it, suspend it when it presents a risk, keep its logs and tell people it is there¹. Chapter 18 lists the Article 26 deployer duties one by one, and chapter 17 covers the incident side of them (deployer duties: inform the provider, suspend use). This chapter follows one system from the decision to use it to the day it is switched off. At each step it names the decision, the artefact that records it, the stack layer that produces the evidence and the record that lands in the assurance store.

Two terms carry the chapter. A deployer is anyone using an AI system under its authority, except in a personal, non-professional activity; a provider develops a system or model, or has it developed, and places it on the market or puts it into service under its own name or trademark². The labels describe tasks, not kinds of organisation: a bank that fine-tunes a vendor model and ships the result under its own brand can hold both roles for the same system (see When a deployer becomes a provider). Chapter 14 covers the build side; this chapter covers the run. Article references are to the EU AI Act as amended by the Digital Omnibus, as of 2026-09-24, whose high-risk rules apply to Annex III systems from 2 December 2027³; they are illustrative mappings, not a claim of conformity.

Governing deployment is not release engineering. MLOps asks whether a new version can be rolled out and rolled back. AI governance engineering asks who decided the system may serve, on what evidence, which signal rolls it back without a meeting, and what record proves each of those happened.

The deployment lifecycle at a glance

The deployment decision

Start from the use case, not the model

The decision to deploy is a product decision with a governance record. Before any model is compared, write down the business objective and how you will know it was met, the people the system will act on, the decision it informs or makes, and whether AI is the right tool at all: a rules engine, a better form or a search index is sometimes the honest answer.

Then name the negative space: what the system is explicitly not for. Negative space is what later makes function creep detectable, because a use that was never approved has somewhere to be recorded as out of scope. The provider’s intended purpose (the use it designed and documented the system for, including the context and conditions of use²) is the outer boundary; your negative space sits inside it.

Classify the use at intake, the workflow described in chapter 06: risk tier, regulatory exposure, data sensitivity and autonomy. The classification decides which of the controls in this chapter apply. An Annex III use triggers the deployer duties of Article 26 ¹ and, for public bodies, private entities providing public services, and credit-scoring and life- and health-insurance pricing uses, the Article 27 fundamental-rights impact assessment (FRIA) before first use⁴. A chatbot or a generator of synthetic content triggers the Article 50 transparency duties, which apply from 2 August 2026⁵; a generative system already on the market before that date has until 2 December 2026 to meet the Article 50(2) marking duty (Article 111(4))⁸. The prohibited practices of Article 5 bind deployers as well as providers, including the ban on systems that exploit vulnerabilities due to age, disability or a specific social or economic situation⁶.

Set performance and explainability requirements first

Requirements written after a vendor demonstration are fitted to the demonstration. Set them before you look at candidates:

• Metrics that match the harm. For a triage classifier, false negatives on urgent cases; for a generative assistant, groundedness and the refusal of out-of-scope requests. An average accuracy figure is rarely the metric that matches a harm.
• Per-group floors. A threshold per population the system acts on, so a good average cannot hide a group the system fails.
• Go/no-go thresholds that the go-live review will apply, each traced to the failure mode it stands for, which is the discipline set out in the limits of the eval gate.
• The explanation the use needs. A reason code for an adverse decision, a cited source for a generated answer, or an interpretable model where the decision must be contestable. When a decision based on the output of most Annex III systems has legal or similarly significant effects on a person, that person has a right to a clear and meaningful explanation from the deployer of the system’s role in the decision⁷. Designing for that answer is cheaper than retrofitting it; chapter 16 covers the methods.

Check the data and the people

Data. Does the input data exist at the coverage and quality the use needs, with a supply that will last, and a lawful basis for this purpose? Where the deployer controls the input data of a high-risk system, it must ensure the data is relevant and sufficiently representative for the intended purpose¹.

People. Oversight must be assigned to people with the necessary competence, training and authority, and the support to use it¹. Article 4, reworded by the Omnibus, asks providers and deployers to support the AI literacy of their staff⁸. Before a high-risk system is used at work, an employer-deployer must inform workers’ representatives and the affected workers¹. Readiness also means the authority to pause: an operator who sees the system misbehave must be allowed to stop it without first asking permission.

The Deployment Decision Record

Pull these answers into one artefact, the Deployment Decision Record (DDR), committed next to the system’s code and referenced from its registry entry (layer 02). Its floors become the thresholds of the eval gate (layer 03); its negative space becomes the scope the runtime watches (layer 04). The templates page has a JSON Schema for the deployment decision and the retirement runbook. An illustrative excerpt:

{ "ddr_id": "ddr-csa-01-v1", "system": "csa-01",
  "objective": "resolve tier-1 refund queries without an agent",
  "not_for": ["credit decisions", "complaints about staff"],
  "risk_tier": "limited", "obligations": ["EU AI Act Art. 50", "GDPR Art. 35"],
  "floors": { "groundedness": 0.95, "out_of_scope_refusal": 0.98 },
  "retire_if": ["deflection below business case for two quarters"],
  "owner": "team-support-platform", "decision": "proceed-to-selection" }

Choosing the model

Evaluate on your task, your data and your users

A public benchmark answers “how good is this model at that benchmark?”. The deployer’s question is “how good is it at our task, for our users, under our constraints?”. Build a selection eval: a set of representative cases drawn from traffic you may lawfully use, labelled with the answer you want, stratified by the groups and edge cases the DDR names, and scored on the metrics that match the harm. Make it large enough to show the per-group difference you care about. Run every candidate through the same harness with the same prompts, retrieval and guardrails, and keep the results: the selection eval becomes the first version of the regression suite behind the eval gate.

What public benchmarks and leaderboards cannot tell you

• Contamination. Test items leak into training data. When researchers wrote a fresh set of grade-school maths problems matched to a popular benchmark, several model families lost up to 8% in accuracy, a sign of overfitting to the public set⁹.
• Selective disclosure. A ranking reflects what providers choose to submit. An analysis of a widely used arena-style leaderboard found that a few providers tested many private variants before release and could retract scores if they chose¹⁰.
• Goodhart. Once a benchmark matters commercially, models are tuned to it. The pressure that makes an eval gate Goodhartable applies with more force to a public number you do not control.

Use benchmarks to build a shortlist, never to make the decision.

Count the whole cost, including energy

The cost of a model is the licence or API fee plus inference compute, integration, evaluation and monitoring, the running cost of the governance stack around it (see the cost of the stack) and the cost of leaving. Energy and carbon belong on the same ledger, and the model choice is where most of them are decided. One study measured multi-purpose generative architectures as orders of magnitude more expensive per inference than task-specific systems on the same tasks, even controlling for model size¹¹. The measurement boundary matters: one provider put the median text prompt of its assistant at 0.24 Wh, counting idle capacity and data-centre overhead as well as the accelerators¹²; a figure that leaves those out is not comparable. At the level of the whole sector, the IEA projects data-centre electricity use to more than double to around 945 TWh by 2030¹³. The AI Act asks GPAI providers to document the known or estimated energy consumption of their models¹⁴; nothing in it measures your inference footprint, so the deployer has to.

Record the choice

The model selection record lists the candidates, their selection-eval results, the cost and energy estimates, the reasons for the choice, the conditions that would reopen it (a price change, a new vendor version, a floor breached in production) and the exit plan. It is filed against the registry entry, so “why this model?” is answered by a query a year later, when the people who chose have moved on.

Model types and deployment options

The model type changes the failure modes. The hosting option changes who can see and stop what. The adaptation technique changes what you must re-test and, sometimes, your legal role. The tables below describe each option, and the matrix at the end names the control each combination adds. Treat all of it as illustrative, not a claim of conformity: the controls a real deployment needs follow from its risk tier, its obligations and its failure modes.

Model type changes the control set

The largest split is classic against generative. Classic models fail quietly, through calibration, drift and group error gaps; generative models fail loudly, through content. Proprietary against open-weight is mostly a question of who produces the evidence: with an API you collect it from the vendor (model card, AIBOM if offered, change notices), with open weights you produce almost all of it yourself (hashes, scans, evals, red-team results, runtime logs; see third-party and procured AI).

Where it runs

How it is adapted

The model-type by deployment-option matrix

Read each cell as the one control the combination adds on top of its row and its column.

Build, buy or adapt

Three routes, three evidence burdens

The less of the model you own, the more of your control budget moves from testing it to bounding it and evidencing the supplier; the more you own, the more of the evidence is yours to produce.

When a deployer becomes a provider

Article 25(1) makes a distributor, importer, deployer or other third party the provider of a high-risk AI system, with the provider’s obligations, when it puts its name or trademark on a high-risk system already on the market; makes a substantial modification to a high-risk system so that it remains high-risk; or changes the intended purpose of a system, including a general-purpose one, so that it becomes high-risk¹⁵. A substantial modification is a change not foreseen in the provider’s initial conformity assessment that affects compliance with the high-risk requirements or changes the intended purpose². The initial provider then stops being the provider of that system but must cooperate; the Omnibus extended that duty to cover technical documentation, known limitations and failure modes, and targeted technical access for testing and validation⁸.

General-purpose models have their own test. The Commission’s guidelines on GPAI obligations (content approved on 18 July 2025, adopted as C(2025) 7719 final on 19 November 2025) ¹⁶ treat an actor that modifies or fine-tunes a GPAI model as the provider of the modified model only in exceptional cases, with an indicative criterion: the modification uses more than one third of the original model’s training compute. The documentation, copyright-policy and training-summary obligations then cover the modification, not the whole model¹⁷; where the original model has systemic risk, the guidelines presume the modified model has it too, with the full systemic-risk duties¹⁶. The criterion is indicative, and how it applies to distillation or repeated adapter training is a question to put to counsel (verify). The engineering consequence is plain either way: log the compute of every fine-tune as an artefact, because the question will be asked.

Open-weight licences

Open weights are not one licence, and “open weight” is not “open source”. Each family asks something different of a deployer:

Three examples show the range. Apache 2.0 adds an express patent grant to its permissions¹⁹. The AGPL requires that users interacting with a modified version over a network be offered its source²⁰, which reaches serving stacks. OpenRAIL-style licences grant open access but attach use restrictions that must pass to every derivative and redistribution²¹. Custom community licences go further: the Llama 3.1 licence incorporates an acceptable-use policy, requires “Built with Llama” attribution, asks that derived models distributed to others carry “Llama” at the start of their name, and requires licensees whose products had more than 700 million monthly active users on the release date to request a separate licence²². The controls follow: a licence check as a policy gate (layer 01), licence, acceptable-use version and file hash in the AIBOM (layer 02), hash verification of every weight file before load, and a scan of serialised model files before they reach a runtime. The full clause and licence checklist is on the contracts page.

Owning the model: the upside and the burden

Building or heavily adapting your own model buys control: over versions and retirement dates, over customisation, over where data lives, and over the evidence itself, since you can red-team weights you hold. It also buys the provider’s burden if you place the system on the market, and three operational risks that API customers mostly rent away. Weight security: the weights are an asset to protect, from theft of the files and from extraction through the inference API, which NIST’s taxonomy of adversarial attacks and MITRE ATLAS both catalogue²³²⁴; rate limits, query-pattern monitoring and least-privilege access to weight storage are the controls. Maintenance: patching, re-validation and base-model upgrades become your calendar. Key-person risk: a model only two people can retrain is a continuity risk with names attached.

Liability, insurance and risk transfer

The new Product Liability Directive brings software, including AI systems, within the definition of a product; Member States must transpose it by 9 December 2026, and it applies to products placed on the market after that date²⁵. For builders that place systems on the market, defect liability becomes a design input; for deployers, the contract decides what recourse they have against the supplier. A market for AI-specific insurance exists, including cover for losses from model errors offered to AI vendors and to the organisations that deploy their systems²⁶. Check existing cyber, technology errors-and-omissions and liability policies for AI exclusions with your broker before assuming cover. Where an insurer asks for evidence of controls, the underwriting questionnaire becomes one more consumer of the assurance store. And risk transferred is not risk reduced: the residual risk left after caps, exclusions and deductibles belongs in the risk register (see chapter 13).

Vendor contracts and licence terms

When you deploy a system you did not build, the contract is a control surface, and the place where the organisation’s third-party AI policy (chapter 12) becomes enforceable. It decides whether you may test the system, whether you hear when it changes, where your data goes and how you leave. The AI Act requires a written agreement between the provider of a high-risk system and third parties that supply its components, tools and services, specifying the information, capabilities, technical access and assistance the provider needs¹⁵. For buyers, the EU model contractual clauses for AI procurement (MCC-AI), updated on 5 March 2025 in a high-risk and a light version with a commentary, give a reference text drafted for public organisations; it is the latest version as of 2026-09-24²⁷. ISO/IEC 42001 Annex A.10 and the NIST AI RMF (GOVERN 6, MANAGE 3) name the third-party controls the clauses support²⁸²⁹³⁰.

The clauses that matter most for governance, with the red flag to look for and the evidence to keep (the fallback position for each is on the contracts page):

Two rules turn the table into governance. First, a clause that matters at runtime must become a check: the no-training setting read back from the account, the residency commitment enforced on the inference path, the version pin held in the registry. A clause nothing checks is a hope. Second, the review is one step of the Vendor / Model Due-Diligence Gate, so it re-opens on renewal and on every material change notice. The dataset behind this table, with the risk each clause addresses, a fallback position and the instruments it maps to, is an engineering checklist, not legal advice.

The go-live review

What the review reads

The go-live review reads an evidence pack, not a slide deck: the DDR; the model selection record; eval gate results against the floors; the red-team summary; the impact assessments (a FRIA where Article 27 applies, a DPIA where data protection law requires one, both maintained as in the FRIA-as-Code pattern); the contract review; the monitoring plan with named owners; the rollout plan with its rollback criteria; the communications plan; and the deactivation runbook. When the evidence is the provider’s own (its instructions for use, which must state the system’s capabilities, limitations, oversight measures and maintenance needs¹⁸), the review runs in review mode: it assesses the supplier’s assessment and records, explicitly, what the deployer could not verify. On the provider side, the same evidence came out of release readiness and conformity (chapter 14).

Three outcomes

An approval with conditions is where governance most often turns into theatre, because conditions are easy to grant and easy to forget. Make them code: each condition is a check with a deadline, and if the check has not passed by then the approval lapses and the feature flag closes. Residual risk is accepted by an authority that matches the risk tier, never by the team that wants to ship.

Recorded dissent

Any member of the review can record dissent. The dissent is attached to the decision record, named, and reviewed at the first monitoring review after go-live. It costs nothing when the dissenter is wrong, and when they are right it answers the first question every incident review asks: did anyone see this coming?

Progressive delivery as a control

Progressive delivery limits exposure while evidence accumulates. Each stage is a gate with a criterion set in advance; the stages are borrowed from site reliability engineering, where canarying is defined as a partial, time-limited deployment of a change and its evaluation³¹.

Rollback criteria must be pre-registered: written into the rollout plan before the stage starts, and evaluated by the pipeline, not by a meeting. A criterion invented after the metric moved is a negotiation, not a control. The same applies to changes you did not make: a new vendor model version is a release, and it goes through shadow and canary on your side against the pinned version before it takes traffic. The tools catalogue lists progressive-delivery tools as illustrative examples, not endorsements.

Operating the system

Policies at go-live

The system goes live with the policies that make its use governable. An acceptable-use policy for staff and customers, drawn from the negative space. Role-based training: what the system is for, the limitations its instructions for use declare¹⁸, when to override it and how to report a problem, with completion as a condition of access. Interface aids that support judgement rather than replace it: sources shown, confidence where it is meaningful, and a visible way to reach a person. These are the practical form of the Article 26 duties to use the system as instructed and to give oversight to competent people with authority¹, and of the responsible-use controls in ISO/IEC 42001 Annex A.9²⁸. Oversight itself is designed as in designing human oversight.

Inference-time data governance

Live inputs are data processing, and a deployed system creates new data with every request: prompts, retrieved passages, outputs, logs and feedback. Govern them as you govern training data (see data governance across the stack):

• Minimise what reaches the model, with PII filtering and data-loss prevention in the input guardrail.
• Set retention per data type as code, reconciling the log-retention floor below with the storage-limitation ceiling of data protection law.
• Re-check the lawful basis when the purpose changes; a new use of old logs is a new processing purpose.
• Map transfers when inference, storage or support run in another jurisdiction; chapter 19 treats remote inference as a transfer.
• Plan for data-subject requests that reach prompts, logs, retrieval corpora and fine-tuned weights. Deleting a record from a corpus is a delete; removing its influence from tuned weights may mean retraining, so decide before you tune on personal data. Deployers of high-risk systems use the provider’s information to carry out their DPIA¹; chapter 19 covers the privacy side in depth.

Maintenance calendar and retraining governance

The provider’s instructions for use state the system’s expected lifetime and the maintenance it needs, including how often¹⁸. The deployer’s calendar starts there and adds its own:

Retraining governance rests on one rule: a retrain, a fine-tune, a prompt change, a corpus refresh and a vendor model update are all releases. Each goes through the eval gate and the progressive delivery stages, and each bumps the version in the registry. A threshold change is a change to a control, so it is a reviewed diff with an approver, not an edit on a dashboard. Retraining is triggered by a drift or floor breach, by the calendar, or by a change in the world the DDR describes. The tools catalogue lists monitoring tools as illustrative examples, not endorsements.

Drift: what moves and how to see it

Concept drift is an unforeseeable change in the distribution of the data a model sees over time, and the research on it divides the work into detection, understanding and adaptation³⁴. In production it helps to name what moved:

Labels often arrive late, or never. For classic models, pair input-drift statistics with a delayed performance check when labels land; for generative systems, sample outputs for groundedness scoring and human review. Every drift signal needs a threshold, an owner and a defined consequence: an issue, a retrain, a degraded mode or an incident.

Fairness and quality in production

A system that passed its fairness evals at go-live can drift into unfairness without any code change. Monitor error rates per group against the per-group floors in the DDR, complaint and appeal rates by group, and, for generative systems, groundedness and refusal rates by topic and language. Where law requires a periodic bias audit (New York City’s Local Law 144, for example, requires a bias audit within one year before an automated employment decision tool is used, and a public summary of its results³⁵), the production telemetry is what makes the audit cheap. Chapter 16 covers the metrics; the point here is that they run continuously and feed the same threshold, issue and incident path as every other signal.

Who owns the signal

The rule is the one from chapter 06: you own a signal when you can be paged for it. A critical system needs coverage for every hour it runs, and a signal with no owner is a signal nobody acts on.

Monitoring third parties while you run

A procured system keeps changing after the contract is signed. File every change and deprecation notice against the registry entry, read the SLA reports, snapshot the sub-processor list, and re-open the due-diligence gate on renewal. Keep an alternative model warm in the eval harness so a forced migration starts from evidence, not from a standing start. The NIST AI RMF asks for exactly this: third-party risks and benefits monitored regularly, and pre-trained models monitored as part of the system’s own maintenance³⁰. The operate step of the due-diligence gate turns this into a runbook.

When the provider fails: continuity

Plan for the supplier failing in each of the ways it can: an outage, a rate limit at peak, a quality drop, a withdrawn model, a forced deprecation, or a commercial or legal exit. The fallbacks are a manual process that staff have practised, an alternative model behind the same interface, cached or templated answers, and the degraded modes described below. Recovery needs its own evidence: back up the corpus snapshot and the embeddings, or know that you can rebuild the index; keep the prompts and guardrail configurations under version control; and test the switch. The NIST AI RMF asks for contingency processes for failures in third-party AI systems deemed high-risk²⁹.

Some sectors make this a legal duty. Financial entities under DORA, in application since 17 January 2025³⁶, must keep a register of information on all contractual arrangements for ICT services from third-party providers, and must have exit strategies for ICT services that support critical or important functions³⁷. An AI model reached as a service is likely to count as an ICT service for this purpose (verify for your case). Entities in scope of NIS2 must take measures for business continuity, including backup management and disaster recovery, and for supply-chain security³⁸; whether you are in scope depends on your sector and size (verify).

Benefit realisation

Track the objective in the DDR against what the system delivers, including its inference and operating cost. Under-delivery is a governance signal, not only a business one: a system that costs more to run and govern than it returns is a candidate for re-scoping or retirement, and the benefit review is where that is decided on evidence. A register of live systems with no measured benefit is an inventory of unpriced risk.

Energy footprint reporting

Measure energy per request and in total, with the boundary stated: accelerators, host systems, idle capacity and data-centre overhead¹². On your own hardware, meter it; on a provider’s, ask for it in the contract and record what you received. Convert to carbon with the grid intensity of the region that serves the traffic, report it next to cost in the benefit review, and let it inform the next model choice: when a smaller model passes the selection eval, the energy difference is a reason to prefer it¹¹.

Records retention

Retention is a control, so it is code: a schedule per record type, tamper-evident storage for logs, a legal hold that overrides deletion, and archive formats someone can still read in 10 years. The six-month floor for logs is a minimum; data protection law sets a ceiling for the personal data inside them, and the two are reconciled per data type, not by keeping everything.

Periodic assurance

An audit programme, not an audit

A single audit is a snapshot. An audit programme has a charter, a risk-based cadence tied to the risk tier, and independence proportionate to the stakes: second-line review, internal audit, or an external assessor (see how this relates to certification). Good programmes re-perform rather than read: the auditor re-runs a fairness check or replays a sample of logs instead of accepting the report that says it was done. Every finding gets an owner, a date and a closure test, and the open findings are a live query, not a spreadsheet.

Red teaming on a schedule

Schedule red teaming by risk tier, and aim it at the deployed configuration (prompts, tools, retrieval corpus, guardrails), not only at the model. For a procured system, test at the boundary within the windows the contract allows. The Adversarial Red-Team Suite pattern turns each finding into a regression test, so the next scheduled run proves the fix held.

Threat modelling the deployed system

Decompose the data flows (user, application, retrieval, model, tools, downstream consumers), then enumerate threats per element with a classic category checklist such as STRIDE, extended with the AI-specific attacks that NIST’s adversarial machine learning taxonomy²³, MITRE ATLAS ²⁴ and the OWASP Top 10 for Agentic Applications ⁴² catalogue. The output that matters is the mapping from threat to mitigation to the test that proves the mitigation works:

The site’s threat bridge lines up these threats with the patterns that control them.

Secondary use and downstream harm

Systems are used for more than they were approved for. The AI Act names the concept: reasonably foreseeable misuse is use not in accordance with the intended purpose, but which may result from reasonably foreseeable human behaviour or interaction with other systems². Around it sit function creep (the gradual extension of a system to purposes no one approved), dual use (the same capability serving a harmful purpose), downstream harm (outputs feeding other systems that act on them), feedback loops (outputs shaping the data the next version learns from, as when a risk score decides who is inspected and so decides which cases get labelled) and synthetic recycling (generated outputs reused as training data).

Forecast them before go-live with three cheap techniques. A premortem: assume it is a year later and the system caused harm, then write down how. Abuse cases: misuse stories written next to the user stories, by people paid to think like the abuser. Stakeholder impact mapping: every group the outputs reach, including those who never touch the interface.

Then give the answers a home, which this chapter calls a downstream use register: intended and prohibited uses written as a Policy Card; every consumer of the outputs (systems, teams, partners) recorded against the registry entry; provenance and caveats stamped on outputs so a consumer knows what it is using; and a re-test whenever outputs are used in a new context. At runtime, off-purpose use is a signal like any other: classify traffic against the negative space and alert on what falls outside it. Consumer-facing deployments add one more question, whether children or other vulnerable people will use the system, with age assurance and the Article 5 prohibition on exploiting vulnerabilities as the controls to consider⁶.

External communications

Every deployed system needs a plan for talking to people outside the organisation, written before it is needed. The plan names one owner and a single voice, an approval workflow, and a single source of truth: a transparency page and a plain-language system card generated from the registry, so what you say publicly cannot drift from what is running. Templates are versioned like code. The NIST AI RMF asks that incidents and errors be communicated to relevant AI actors, including affected communities³⁰.

A holding statement is prepared in skeleton before any incident: what happened, stated only as far as it is known; what has been done to contain it; what affected people should do; and when the next update will come. It never speculates about cause. The incident side of communication is developed in chapter 17. At retirement, the same plan sends the sunset notices. And measure the plan: whether notices reached the people they were for, and what the complaint volume did afterwards.

Deactivation, degradation, localisation and retirement

A deactivation policy someone can execute

A deactivation policy names its triggers, its decision authority, the record each decision leaves, how evidence is preserved and the criteria for a safe restart. Triggers come in two kinds. Threshold triggers: a floor breached and not recovered within a set window, a fairness gap above its limit, an incident severity. Legal triggers: the deployer’s own duty to suspend use when it has reason to consider the system presents a risk¹; a provider’s corrective action to withdraw, disable or recall a non-conforming system⁴⁵; action by an authority on an AI system presenting a risk to health, safety or fundamental rights⁴⁶; and a practice newly prohibited. The NIST AI RMF asks for mechanisms, with assigned responsibilities, to supersede, disengage or deactivate systems whose performance or outcomes are inconsistent with intended use³⁰.

Evidence preservation comes first: freeze the logs, apply a legal hold, snapshot the versions. Then stop. Deactivation applies to every kind of system, not only agents: a classifier embedded in a vendor product needs a switch too, whether a feature flag or a fallback path. For agents, the Kill Switch / Circuit Breaker pattern applies, and chapter 23 develops it.

Graduated degradation

Switching off is the last resort, not the only one. Build the intermediate modes in advance as operational toggles³³, and test them:

Localisation by jurisdiction

Launch only where compliance has been shown, and keep the jurisdiction as a policy input rather than a deployment accident: per-jurisdiction rule sets as code, regional instances where residency requires them, and feature flags by region so one market can be switched off without touching the others. Obligations overlap in places (the AI Act’s six-month log floor and the six-month operator log retention in China’s voluntary TC260 framework¹⁴¹) and diverge in many others; see chapter 21 and the regulatory map.

Retirement and decommissioning

Design retirement in from the start: the DDR already names the conditions under which the system is retired. Triggers include a benefit shortfall, a vendor deprecation, a replacement and a legal event. The NIST AI RMF warns that irregular or indiscriminate termination can itself increase risk²⁹, so retirement is a runbook, not a deletion:

1. Dependency analysis. Who consumes the outputs? The downstream use register answers it.
2. Fallback and transition. Users move to the replacement or the manual process, with training.
3. Sunset notices. Customers, partners and affected people hear before the date, not after.
4. Final evidence snapshot. Cards, evals, decisions and logs are archived per the retention schedule.
5. Archive or dispose. Weights, corpora and logs are kept or destroyed as licence, lawful basis and retention decide.
6. Revoke identities and credentials. Every non-human identity the system held is revoked.
7. Retire, do not delete, the registry entry. Its status reads retired, with the date and the decision record.
8. Confirm it is gone. Shadow-AI Discovery checks that no copy still runs.

One system from decision to retirement

csa-01, the assistant that chapter 04 follows through the five layers, also runs through this chapter. The artefacts it leaves, all illustrative:

Maps to: EU AI Act Art. 4, 5, 13, 25, 26, 27, 50, 86 · GDPR Art. 33, 34, 35 · ISO/IEC 42001 Annex A.6.2.5, A.6.2.6, A.9, A.10 · NIST AI RMF (GOVERN 1.7, GOVERN 6, MANAGE 2.4, MANAGE 3, MANAGE 4) · DORA Art. 28 · NIS2 Art. 21 · all five layers. Mappings are illustrative, not a claim of conformity.

What you can do this week

1. Write the DDR for your riskiest live system, retroactively if you must, including its negative space and retirement conditions, and link it from the registry entry.
2. Check five clauses in your largest AI contract: use of your data for training, change and deprecation notice, incident notice, audit and evaluation access, and exit. File each gap as a risk.
3. Pre-register the rollback criteria for your next model, prompt or vendor-version change, and wire one of them as an automatic canary check.
4. Compare log retention on every high-risk or likely high-risk system against the six-month floor and your data-protection ceiling.
5. Build one degraded mode (advice-only or grounded-only) behind a flag, and test that it works.

Sources

1. [1] Regulation (EU) 2024/1689 (AI Act), consolidated text of 2026-07-27, Art. 26 (deployer obligations: 26(1) use per the instructions; 26(2) oversight by competent persons with authority; 26(4) relevant and representative input data; 26(5) monitor, suspend and inform, serious incidents to the provider first; 26(6) logs kept at least six months; 26(7) inform workers; 26(8) public-authority registration; 26(9) DPIA; 26(11) inform affected persons). Publications Office of the EU (EUR-Lex). 2026-07-27. https://eur-lex.europa.eu/eli/reg/2024/1689/2026-07-27/eng#art_26 (verified: primary)
2. [2] Regulation (EU) 2024/1689 (AI Act), consolidated text of 2026-07-27, Art. 3 definitions: (3) provider, (4) deployer, (12) intended purpose, (13) reasonably foreseeable misuse, (23) substantial modification. Publications Office of the EU (EUR-Lex). 2026-07-27. https://eur-lex.europa.eu/eli/reg/2024/1689/2026-07-27/eng#art_3 (verified: primary)
3. [3] “AI Omnibus enters into force” (Reg. (EU) 2026/1744, in force 2026-07-27; Annex III high-risk rules from 2 Dec 2027; Annex I from 2 Aug 2028). European Commission. 2026-07-27. https://digital-strategy.ec.europa.eu/en/news/ai-omnibus-enters-force (verified: primary)
4. [4] Regulation (EU) 2024/1689 (AI Act), consolidated text of 2026-07-27, Art. 27(1) (FRIA before first use by public bodies, private entities providing public services and deployers of Annex III points 5(b) and (c)). Publications Office of the EU (EUR-Lex). 2026-07-27. https://eur-lex.europa.eu/eli/reg/2024/1689/2026-07-27/eng#art_27 (verified: primary)
5. [5] “Safer and more transparent AI” (Art. 50 transparency obligations apply from 2 Aug 2026). European Commission. 2026-08-02. https://commission.europa.eu/news-and-media/news/safer-and-more-transparent-ai-2026-08-02_en (verified: primary)
6. [6] Regulation (EU) 2024/1689 (AI Act), consolidated text of 2026-07-27, Art. 5(1)(b) (prohibition on exploiting vulnerabilities due to age, disability or a specific social or economic situation; applies to placing on the market, putting into service and use). Publications Office of the EU (EUR-Lex). 2026-07-27. https://eur-lex.europa.eu/eli/reg/2024/1689/2026-07-27/eng#art_5 (verified: primary)
7. [7] Regulation (EU) 2024/1689 (AI Act), consolidated text of 2026-07-27, Art. 86(1) (right to a clear and meaningful explanation from the deployer of the role of an Annex III system in a decision, except point 2 systems). Publications Office of the EU (EUR-Lex). 2026-07-27. https://eur-lex.europa.eu/eli/reg/2024/1689/2026-07-27/eng#art_86 (verified: primary)
8. [8] Regulation (EU) 2024/1689 (AI Act), consolidated text of 2026-07-27, as amended by Regulation (EU) 2026/1744 (Art. 4 replaced: providers and deployers support AI literacy; Art. 25(2) cooperation extended to technical documentation, known limitations and failure modes and targeted technical access; Art. 25(4) revised; Art. 26 unchanged; Art. 111(4) transitional period for Art. 50(2) marking to 2 Dec 2026 for generative systems placed on the market before 2 Aug 2026). Publications Office of the EU (EUR-Lex). 2026-07-27. https://eur-lex.europa.eu/eli/reg/2024/1689/2026-07-27/eng (verified: primary)
9. [9] “A Careful Examination of Large Language Model Performance on Grade School Arithmetic” (GSM1k; accuracy drops of up to 8% against GSM8k; systematic overfitting in several model families) (arXiv 2405.00332). Zhang et al. 2024-05-01. https://arxiv.org/abs/2405.00332 (verified: primary)
10. [10] “The Leaderboard Illusion” (undisclosed private testing of multiple variants and score retraction on Chatbot Arena) (arXiv 2504.20879). Singh et al. 2025-04-29. https://arxiv.org/abs/2504.20879 (verified: primary)
11. [11] “Power Hungry Processing: Watts Driving the Cost of AI Deployment?” (multi-purpose generative architectures orders of magnitude more expensive per inference than task-specific systems, controlling for parameters) (arXiv 2311.16863; FAccT ‘24). Luccioni, Jernite, Strubell. 2023-11-28. https://arxiv.org/abs/2311.16863 (verified: primary)
12. [12] “Measuring the environmental impact of delivering AI at Google Scale” (median Gemini Apps text prompt 0.24 Wh; boundary includes host energy, idle capacity and data-centre overhead) (arXiv 2508.15734). Elsworth et al., Google. 2025-08-21. https://arxiv.org/abs/2508.15734 (verified: primary)
13. [13] Energy and AI, executive summary (data-centre electricity 415 TWh in 2024, around 945 TWh by 2030). International Energy Agency. 2025-04. https://www.iea.org/reports/energy-and-ai/executive-summary (verified: primary)
14. [14] Regulation (EU) 2024/1689 (AI Act), consolidated text of 2026-07-27, Annex XI, Section 1, point 2(e) (GPAI technical documentation: known or estimated energy consumption of the model). Publications Office of the EU (EUR-Lex). 2026-07-27. https://eur-lex.europa.eu/eli/reg/2024/1689/2026-07-27/eng#anx_XI (verified: primary)
15. [15] Regulation (EU) 2024/1689 (AI Act), consolidated text of 2026-07-27, Art. 25 (value chain: 25(1)(a) name or trademark, (b) substantial modification, (c) changed intended purpose; 25(2) cooperation of the initial provider; 25(4) written agreement with third-party suppliers). Publications Office of the EU (EUR-Lex). 2026-07-27. https://eur-lex.europa.eu/eli/reg/2024/1689/2026-07-27/eng#art_25 (verified: primary)
16. [16] Commission Guidelines on the scope of the obligations for providers of general-purpose AI models established by Regulation (EU) 2024/1689 (C(2025) 7719 final; content approved 18 Jul 2025; paras. 67 to 68: a modified systemic-risk model is presumed to have systemic risk). European Commission. 2025-11-19. https://digital-strategy.ec.europa.eu/en/library/guidelines-scope-obligations-providers-general-purpose-ai-models-under-ai-act (verified: primary)
17. [17] Guidelines on obligations for general-purpose AI providers, FAQ (modifiers become providers only when the modification uses more than one third of the original model’s training compute; obligations limited to documenting the modification). European Commission. 2025. https://digital-strategy.ec.europa.eu/en/faqs/guidelines-obligations-general-purpose-ai-providers (verified: primary)
18. [18] Regulation (EU) 2024/1689 (AI Act), consolidated text of 2026-07-27, Art. 13(3) (instructions for use: capabilities and limitations of performance; pre-determined changes; human oversight measures; expected lifetime and maintenance measures, including their frequency; log collection). Publications Office of the EU (EUR-Lex). 2026-07-27. https://eur-lex.europa.eu/eli/reg/2024/1689/2026-07-27/eng#art_13 (verified: primary)
19. [19] Apache License, Version 2.0 (section 3, grant of patent licence). Apache Software Foundation. 2004-01. https://www.apache.org/licenses/LICENSE-2.0 (verified: primary)
20. [20] GNU Affero General Public License v3 (section 13, remote network interaction). Free Software Foundation. 2007-11-19. https://www.gnu.org/licenses/agpl-3.0.html (verified: primary)
21. [21] “OpenRAIL: Towards open and responsible AI licensing frameworks” (use-based restrictions that must be adopted by redistributions and derivatives). Hugging Face. 2022-08-31. https://huggingface.co/blog/open_rail (verified: primary)
22. [22] Llama 3.1 Community License Agreement (Acceptable Use Policy incorporated by reference; “Built with Llama” attribution; “Llama” at the start of distributed derived model names; separate licence above 700 million monthly active users on the release date). Meta. 2024-07-23. https://github.com/meta-llama/llama-models/blob/main/models/llama3_1/LICENSE (verified: primary)
23. [23] NIST AI 100-2 E2025, Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations. NIST. 2025-03. https://csrc.nist.gov/pubs/ai/100/2/e2025/final (verified: primary)
24. [24] MITRE ATLAS: adversarial tactics and techniques knowledge base for AI systems. MITRE. 2026. https://atlas.mitre.org/ (verified: primary)
25. [25] Directive (EU) 2024/2853 on liability for defective products (software within the definition of product; transposition by 9 Dec 2026; applies to products placed on the market or put into service after that date). Publications Office of the EU (EUR-Lex). 2024-10-23. https://eur-lex.europa.eu/eli/dir/2024/2853/oj/eng (verified: primary)
26. [26] aiSure AI insurance (cover for losses from AI model errors, for AI vendors and corporate adopters). Munich Re. 2026. https://www.munichre.com/en/solutions/for-industry-clients/insure-ai.html (verified: primary)
27. [27] Updated EU AI model contractual clauses (MCC-AI high-risk and light versions, with commentary; update of the 2023 clauses). Community of Practice on Public Procurement of AI, Public Buyers Community (European Commission). 2025-03-05. https://public-buyers-community.ec.europa.eu/communities/procurement-ai/resources/updated-eu-ai-model-contractual-clauses (verified: primary)
28. [28] ISO/IEC 42001:2023, Annex A control titles (A.6.2.5 AI system deployment; A.6.2.6 operation and monitoring; A.9 use of AI systems; A.10 third-party and customer relationships), referenced by identifier only. ISO/IEC (titles checked via a secondary listing). 2023. https://www.iso.org/standard/42001 (verified: secondary)
29. [29] NIST AI RMF Playbook, GOVERN (1.7 decommissioning and phasing out safely; 6.1 third-party risk policies; 6.2 contingency for failures in high-risk third-party systems). NIST. 2023. https://airc.nist.gov/airmf-resources/playbook/govern/ (verified: primary)
30. [30] NIST AI RMF Playbook, MANAGE (2.4 supersede, disengage or deactivate; 3.1 third-party risks monitored; 3.2 pre-trained models monitored; 4.1 post-deployment monitoring plans; 4.3 incidents communicated, including to affected communities). NIST. 2023. https://airc.nist.gov/airmf-resources/playbook/manage/ (verified: primary)
31. [31] The Site Reliability Workbook, ch. 16 “Canarying Releases” (“a partial and time-limited deployment of a change in a service and its evaluation”). Google (O’Reilly). 2018. https://sre.google/workbook/canarying-releases/ (verified: primary)
32. [32] “BlueGreenDeployment” (two identical production environments; switch back on failure). Martin Fowler. 2010-03-01. https://martinfowler.com/bliki/BlueGreenDeployment.html (verified: primary)
33. [33] “Feature Toggles (aka Feature Flags)” (release, experiment, ops and permissioning toggles; ops kill switches for graceful degradation). Pete Hodgson, martinfowler.com. 2017-10-09. https://martinfowler.com/articles/feature-toggles.html (verified: primary)
34. [34] “Learning under Concept Drift: A Review” (IEEE TKDE 31(12); detection, understanding and adaptation) (arXiv 2004.05785). Lu et al. 2018. https://arxiv.org/abs/2004.05785 (verified: primary)
35. [35] Automated Employment Decision Tools (NYC Local Law 144 of 2021: bias audit within one year before use, public summary, candidate notices; enforced from 5 Jul 2023). NYC Department of Consumer and Worker Protection. 2023. https://www.nyc.gov/site/dca/about/automated-employment-decision-tools.page (verified: primary)
36. [36] Digital Operational Resilience Act (DORA): in application since 17 Jan 2025; register of information; ICT third-party risk. EIOPA. 2025. https://www.eiopa.europa.eu/digital-operational-resilience-act-dora_en (verified: primary)
37. [37] Regulation (EU) 2022/2554 (DORA), Art. 28(3) register of information on ICT third-party arrangements and Art. 28(8) exit strategies for ICT services supporting critical or important functions. Publications Office of the EU (EUR-Lex). 2022-12-14. https://eur-lex.europa.eu/eli/reg/2022/2554/oj/eng#art_28 (verified: primary)
38. [38] Directive (EU) 2022/2555 (NIS2), Art. 21(2)(c) business continuity, backup management, disaster recovery and crisis management, and (d) supply-chain security. Publications Office of the EU (EUR-Lex). 2022-12-14. https://eur-lex.europa.eu/eli/dir/2022/2555/oj/eng#art_21 (verified: primary)
39. [39] Regulation (EU) 2024/1689 (AI Act), consolidated text of 2026-07-27, Art. 19(1) (providers keep automatically generated logs for at least six months). Publications Office of the EU (EUR-Lex). 2026-07-27. https://eur-lex.europa.eu/eli/reg/2024/1689/2026-07-27/eng#art_19 (verified: primary)
40. [40] Regulation (EU) 2024/1689 (AI Act), consolidated text of 2026-07-27, Art. 18(1) (providers keep documentation for 10 years after placing on the market or putting into service). Publications Office of the EU (EUR-Lex). 2026-07-27. https://eur-lex.europa.eu/eli/reg/2024/1689/2026-07-27/eng#art_18 (verified: primary)
41. [41] AI Safety Governance Framework 3.0, §5.3 operators’ guidelines (logs kept for at least six months and audited; voluntary). TC260 / CAC. 2026-09-14. https://www.cac.gov.cn/rootimages/uploadimg/1791137114683961/1791137114683961.pdf (verified: primary)
42. [42] Top 10 for Agentic Applications 2026 (ASI01 to ASI10). OWASP GenAI Security Project. 2025-12-09. https://genai.owasp.org/resource/owasp-top-10-for-agentic-applications-for-2026/ (verified: primary)
43. [43] Regulation (EU) 2016/679 (GDPR), Art. 33 (notification of a personal data breach to the supervisory authority without undue delay and, where feasible, within 72 hours). Publications Office of the EU (EUR-Lex). 2016-04-27. https://eur-lex.europa.eu/eli/reg/2016/679/oj/eng#art_33 (verified: primary)
44. [44] Regulation (EU) 2016/679 (GDPR), Art. 34(1) (communication of a breach likely to result in a high risk to the data subject without undue delay). Publications Office of the EU (EUR-Lex). 2016-04-27. https://eur-lex.europa.eu/eli/reg/2016/679/oj/eng#art_34 (verified: primary)
45. [45] Regulation (EU) 2024/1689 (AI Act), consolidated text of 2026-07-27, Art. 20(1) (providers take corrective action: bring into conformity, withdraw, disable or recall; inform distributors and deployers). Publications Office of the EU (EUR-Lex). 2026-07-27. https://eur-lex.europa.eu/eli/reg/2024/1689/2026-07-27/eng#art_20 (verified: primary)
46. [46] Regulation (EU) 2024/1689 (AI Act), consolidated text of 2026-07-27, Art. 79 (procedure for AI systems presenting a risk to health, safety or fundamental rights). Publications Office of the EU (EUR-Lex). 2026-07-27. https://eur-lex.europa.eu/eli/reg/2024/1689/2026-07-27/eng#art_79 (verified: primary)