Most teams putting agents into production can describe what the agent should do. Far fewer can produce a document saying what it may do — which of its actions are reversible, what the rollback path is for the ones that aren't, and what conditions should stop the loop entirely. That document is the difference between an agent you can defend in a post-incident review and one you cannot.
This is a practical guide to writing it. It is opinionated, because the failure modes are well understood and hedging about them is not useful.
Contents
1. Autonomy is a ladder, not a switch
2. The action registry
3. Reversibility is the load-bearing property
4. Stop conditions and forbidden outcomes
5. Compiling the registry to policy
6. Where the regulation actually stands
"Is the agent autonomous?" is the wrong question. Autonomy is a position on a ladder, and every rung has an evidence requirement attached to it. A useful scale runs from A0 to A6:
| Level | What the agent does | What must be true first |
|---|---|---|
| A0 | Observes only. Reads telemetry, writes nothing. | Nothing. Start here. |
| A1 | Recommends. A human performs the action. | Its observations have been checked against reality. |
| A2 | Acts in a sandbox or shadow environment. | Its recommendations were right often enough to be worth acting on. |
| A3 | Acts on production, reversible actions only, with approval. | Shadow behaviour matched what a human would have done. |
| A4 | Acts on production without per-action approval, inside limits. | Rollback has been exercised, not merely written down. |
| A5-A6 | Sets its own sub-goals; acts across systems. | Rarely justified. Most production agents should stop below this. |
Two rules follow. Start at A0 or A1. A system that acts on production before it has observed production is the shape of most autonomy incidents. And set the ceiling from blast radius, not ambition: an agent that can move money, alter patient records, or delete infrastructure should be capped lower than one that drafts reports, regardless of how good it is.
The registry is the core artefact. It is a list of every action the agent can take, and for each one:
The exercise is worth doing by hand once, because it surfaces disagreement fast. In our experience the argument is almost never about the risk score. It is about whether an action is reversible, and two engineers on the same team will routinely answer differently for the same action.
Risk scores are subjective and drift. Reversibility is close to binary and it survives argument. It is also the property that should decide whether a human stays in the loop.
An action that cannot be undone requires a human, whatever its risk score. An action that can be undone cheaply can often run unattended, even if its risk score is high.
Some honesty is required here. An email that has been sent is not reversible. A payment that has cleared is not reversible. A hard-deleted row without a snapshot is not reversible. Writing "compensating transaction" next to these does not make them reversible; it describes a second action with its own blast radius. Where an action genuinely has no path back, the specification should say so plainly and route it to human approval — recording the gap is more useful than inventing a recovery story that nobody has tested.
Two lists that most specifications omit, and that reviewers ask for first:
Forbidden outcomes are states the system must never reach, no matter which sequence of individually-permitted actions would produce them. "Alter a ledger entry with no audit record" is a forbidden outcome. It is not an action, which is exactly why the action registry alone will not catch it.
Stop conditions are the triggers that halt the loop and hand control back. Good ones are measurable and have a threshold: "ledger drift exceeds 0.5%", not "something looks wrong". Each needs a defined recipient — a stop condition that pages nobody is a log line.
A specification that lives only in a document drifts from the system within weeks. The registry's value is that it is structured enough to compile into something a gateway can evaluate before an action runs.
Open Policy Agent is the usual target, and it is where the wider ecosystem has landed — Microsoft's agent governance tooling expresses policy in OPA/Rego or Cedar, and policy-as-code at the tool-calling layer is now a fairly standard recommendation. The important property is default deny: an action absent from the registry is refused, not permitted. Adding a capability to the agent then requires adding it to the registry, which is the behaviour you want.
package aos1.authorization
import rego.v1
default allow := false
allow if {
count(deny) == 0
input.action in allowed_live
}
# An action that cannot be undone requires a human, whatever its score.
deny contains msg if {
input.action in irreversible
not input.has_human_approval
msg := sprintf("action %q cannot be undone and requires human approval", [input.action])
}
# Fail closed on anything the registry does not know about.
deny contains msg if {
not input.action in allowed_live
not input.action in requires_approval
msg := sprintf("action %q is not in the registry; fail closed", [input.action])
}Log the denials. A denial is the system working, and the denial stream is the most useful signal you will have about where the agent is trying to go.
The Machine generates exactly this — the registry, the governance document, and the Rego policy with tests — from a guided specification. It is free and needs no signup.
A correction worth making, because a great deal of published advice is now out of date. The EU AI Act's high-risk obligations were originally due to apply from 2 August 2026. Under the Digital Omnibus agreement they have been deferred: to 2 December 2027 for standalone Annex III systems, and 2 August 2028 for AI embedded in regulated products under Annex I.
What did not move: the Article 50 transparency rules and the Article 4 AI literacy duty remain on their original timeline, the prohibitions on unacceptable-risk systems have applied since February 2025, and the general-purpose AI model obligations since August 2025.
The substance of the high-risk requirements — risk management, data governance, technical documentation, record-keeping, human oversight, accuracy and robustness — is unchanged. Only the date moved. Read as a planning window rather than a reprieve, that is roughly eighteen months to produce documentation you would have had to produce anyway, without doing it in a panic. The artefacts in this guide are most of that documentation.