Graduated Automation: How to Scale AI Without Losing Control

A procedure that can run without approval needs a smaller job than "fix the incident."

It needs a defined action, a set of conditions in which that action is permitted, and a way to stop when those conditions no longer hold. A convincing diagnosis does not supply any of those things by itself.

That is what I mean by graduated automation: widening a particular procedure's permission to act as evidence justifies it, while retaining the ability to withdraw that permission. Some procedures should never graduate to unattended execution.

The design below is a proposal built around a hypothetical job-processing service. It is not a description of a Microsoft system or a report of production results. The public sources support the underlying control principles; they do not demonstrate that this particular design improves reliability or reduces cost.

Separate reasoning from permission

A deterministic script, a workflow with an AI decision point, and an agent choosing its next tool call describe different ways to perform work. They do not tell us how much authority the work should have.

A script can reliably delete the wrong data. An exploratory agent can investigate an unfamiliar failure without having permission to change anything. Replacing model reasoning with a fixed procedure may make some behaviour easier to test, but it does not make the procedure safe to run unattended.

I would track execution method and execution authority separately. A procedure might be allowed only to recommend an action, to execute a specific human-approved action, or to act within a pre-approved scope. An explicit suspended state blocks execution when the evidence or controls no longer justify it.

That distinction is consistent with OWASP's guidance on excessive agency. It separates excessive functionality, permissions, and autonomy, and recommends limiting each. It also calls for human approval of high-impact actions and authorization enforced in downstream systems rather than left to the model. [1]

Even read-only access needs limits. An investigation can expose sensitive records or overload a dependency through repeated queries. "Cannot write" is useful, but it is not a complete access policy.

Start with one procedure

Consider a stalled worker in the hypothetical service. A proposed procedure would restart that worker after confirming that the rest of the service can carry its load.

Initially, the agent can inspect permitted health signals and recommend a restart. It cannot execute one. The team can compare its recommendations with independently reviewed cases, including cases where a restart would be the wrong response. This observation phase tests diagnosis and decision boundaries; it does not demonstrate what would have happened if every recommendation had been executed.

The next step could allow a human-approved restart. Approval attaches to the selected worker, the exact operation and parameters, and a specific version of the procedure. It is single-use and expires. If the request changes, the earlier approval does not travel with it.

Only after reviewing the evidence would the owner consider unattended execution for a narrow class of cases. The procedure would check that the worker is an allowed target, spare capacity is available, and in-flight jobs can be handled without unacceptable loss or duplication. It would refuse to act during a conflicting deployment or another recovery operation.

If those conditions cannot be established, the procedure stays recommendation-only. A missing health signal is not permission to assume the service is healthy.

Restarting the worker does not grant permission to restart its neighbours, change its configuration, or repair whatever the agent discovers next. Each of those is another decision about scope.

Decide what evidence earns permission

A streak of successful runs is a poor graduation rule on its own. The runs may all cover the same easy case. A procedure can look dependable until its first encounter with stale telemetry or a partially completed operation.

Before widening permission, I would ask the responsible owner to review four things:

1. The permitted scope. Which targets and conditions are covered? What is explicitly excluded? What limits apply to concurrent actions and repeated attempts?
2. Evidence at the boundaries. Test expected cases and cases that must be rejected. Include unavailable dependencies, misleading inputs, and interruption between steps. Define success and harmful failure in advance, report failures alongside successes, and state which conditions remain untested.
3. Detection and recovery. Show how the system detects a bad outcome and what it can actually do afterward. Exercise the stop path. Check that the proposed recovery is possible rather than labelling every operation reversible.
4. Accountability and validity. Record who accepts the residual risk, the procedure and policy versions covered, and when the permission must be reviewed. Changes to a model, tool, or dependency should trigger an assessment of whether the existing evidence still applies.

The acceptance criteria should be measurable and chosen before the evaluation. There is no defensible universal success percentage for every procedure. A low-impact, recoverable action and an irreversible change need different evidence. Observing no harmful failures in a finite evaluation does not prove that the failure rate is zero.

Graduation is an approval decision about a documented scope. It should not happen because the agent reports high confidence, or because nobody has complained recently.

Put the permission check outside the model

The model can propose an action. A separate execution service should decide whether that request is currently allowed. The agent must not be able to edit its own policy, approve its own request, or reach the same operation through a broader credential.

For the worker restart, that service would validate the target and parameters, check the current permission state, and enforce concurrency and retry limits. It would recheck the relevant health conditions immediately before execution. If another operation can change those conditions, the design needs coordination or a conditional operation, not just an earlier snapshot that looked safe.

This follows OWASP's recommendation to enforce authorization in downstream systems and validate requests against security policy. [1] A prompt telling the agent to be careful cannot replace that check.

A human approval must pass through the same controls. The reviewer should see the proposed operation, its likely impact, and the evidence behind it, with a way to inspect that evidence independently. Approval should not bypass a hard safety limit.

And a tool returning "success" is not enough to close the case. Check the resulting service state and whether work is progressing again. Keep an audit record linking the proposal to the policy decision, any approval, and the observed outcome. Apply access and retention controls to that record; collecting every raw diagnostic payload can create another data exposure.

Withdrawal needs its own design

Suppose the restart request times out. The worker may have restarted even though the caller received no confirmation. Retrying immediately could interrupt it again.

The procedure needs a defined response to that uncertainty. In this example, it would block further restarts, inspect the actual state if that is still permitted, and alert the responsible operator. It would not let the agent improvise a more invasive repair.

One rejected request need not suspend every procedure in the system. A failed precondition can block the current attempt. Evidence that a procedure's assumptions are wrong, that monitoring has stopped working, or that an action caused unexpected harm may warrant suspending that procedure. Encode those suspension triggers in the execution service before deployment, so withdrawal does not depend on a person noticing an alert.

Suspension also has to reach queued work and the execution service, not just change a label in a dashboard. An operation already in flight needs a safe stopping point or a separate recovery plan. Stopping future actions does not undo an action already taken.

NIST's AI Risk Management Framework includes mechanisms to disengage or deactivate systems in MANAGE 2.4. MANAGE 4.1 addresses post-deployment monitoring, including incident response, recovery, and change management. [2] These are lifecycle controls, not a guarantee that a deployed system will remain safe.

Re-enabling unattended execution should require a deliberate review of what changed, the corrective work, and the new evidence. A timer expiring is not enough.

Count the work that remains

The attraction of this approach is that people can approve a bounded procedure instead of reviewing every repetition. Whether it saves effort depends on what remains inside that boundary and how expensive the exceptions are.

I would track outcomes per procedure and version: eligible cases, attempted actions, verified recoveries, harmful outcomes, and unresolved cases. Rejected actions and human interventions belong in the record too. Otherwise a system that declines the difficult cases can look better than one doing different work.

Measure the human effort spent reviewing requests, investigating failures, maintaining policies, and practising recovery. Separate time waiting for a reviewer from time spent reviewing. A falling escalation percentage can still leave people with more work if total volume rises.

There is no promised month when oversight becomes cheap. The decision is whether the measured benefit justifies the remaining risk and operating cost for this procedure. Leaving a procedure approval-only can be the right outcome.

Before granting unattended execution, I want the owner to finish this sentence: this procedure may take this action on these targets, under these conditions, until this evidence or permission is no longer valid.