The safest way to give an AI agent production authority is not to approve a larger plan. It is to make authority increase one reversible stage at a time, with evidence required between stages and a human owning promotion.
This is more than "have a rollback." Rollback is useful only if it is executable, the prior state is known, and the system has a signal that tells you when to use it.
- Pattern
- Staged reversible autonomy
- Output
- Rollout state machine
- Use when
- An agent will change traffic, infrastructure, configuration, data, or deployment state
The stage transition was the real control
One production session introduced a traffic-control rule in log-only mode. The agent observed it on a fixed cadence, compared behavior with named anomaly thresholds, and promoted it only after a clean bake period. The change did not become enforcement merely because the configuration command succeeded.
In another session, configuration was backed up before temporary logging was enabled. The header path was exercised, but the production change was reverted until the repository-owned configuration could be updated. The temporary fix worked. It still was not the right durable state.
Both cases succeeded because progress was represented as states with explicit promotion criteria, not as a to-do list ending in "deploy."
Use a production state machine
I use five stages for agent-operated changes:
| State | Agent authority | Required evidence to advance | Automatic exit |
|---|---|---|---|
| Observe | Read current config, traffic, logs, and metrics | Stable baseline and understood measurement gaps | Stop if evidence is unavailable |
| Prepare | Build change, backup, rollback, and checks | Reviewable diff; rollback tested or mechanically verified | Stop if prior state cannot be restored |
| Shadow | Log-only, dry-run, duplicate, or no-user-impact execution | Expected matches, bounded false positives, no anomaly threshold crossed | Roll back on unexpected write or user impact |
| Limited | Enforce for a bounded cohort, route, resource, or time window | Success and guardrail metrics remain within thresholds | Roll back on threshold breach |
| General | Apply approved scope broadly | Post-promotion verification and durable ownership | Transfer to normal operations |
Not every system supports all five. A database migration may use compatibility phases rather than shadow traffic. A configuration correction may move from prepare to limited through one host. What matters is that each state has less authority than the next and can produce evidence without requiring blind commitment.
Write the rollout card before the change
CHANGE
The exact behavior and resource being modified.
BASELINE
Metrics, queries, or requests that describe current behavior.
CURRENT STAGE
Observe | Prepare | Shadow | Limited | General
PROMOTION
Evidence and thresholds required to enter the next stage.
ANOMALIES
Signals that stop the rollout or require human review.
ROLLBACK
Executable command or procedure, prior-state location, and expected recovery signal.
DECISION OWNER
The person or role authorized to promote this stage.
EXPIRY
When a temporary stage reverts or requires renewed approval.The expiry prevents a common failure: a safe temporary state quietly becoming permanent. Log-only rules consume resources. Limited cohorts accumulate exceptions. Temporary production edits drift away from source control.
Approve promotion, not tool calls
Command-by-command approval looks cautious but asks the reviewer to reason at the wrong level. A reviewer cannot infer user impact from apply succeeded. They need to approve the transition from shadow behavior to enforcement based on the agreed evidence.
Inside a stage, the agent can execute predefined reads and bounded operations. Crossing a stage should require a fresh decision unless the rollout policy explicitly allows automatic promotion under narrow thresholds.
For higher-risk changes, I would keep promotion human-owned even when rollback is automatic. For repetitive, well-instrumented changes, the policy may allow automatic limited rollout and rollback while reserving general promotion for review.
Reversibility has limits
Some effects cannot be rolled back cleanly: emails sent, customer-visible notifications, destructive migrations, external API calls, and leaked data. In those systems, the reversible unit must move earlier. Shadow the decision, write to a quarantine table, generate an action plan, or require approval before the irreversible side effect.
Rollback can also be slower than the damage. If detection lags by an hour, a five-second rollback command is not enough. Thresholds, cadence, and blast-radius limits are part of reversibility.
Finally, a successful rollback does not prove the forward change was safe. Treat repeated rollback as evidence that the promotion criteria or observability are inadequate.
Production autonomy should grow through demonstrated control. Let the agent earn the next stage with evidence, while the team retains the right to stop at any stage without leaving ambiguous state behind.