Designing Decision Boundaries for AI Autonomy

"Let the agent handle routine cases" is not a rule an engineer can implement without making policy decisions on someone else's behalf. What counts as routine? Which facts must still be true when the action runs? What happens when the system cannot tell?

A decision boundary should answer those questions for a specific action. It separates what the system may execute from what it may only propose, and what it must refuse or pause.

I find familiarity and reversibility useful prompts when writing that boundary. I would not turn them into a four-box permission chart. A familiar request to disclose private information can still be harmful. An unfamiliar request to draft a private note may be harmless enough to attempt. Consequence, authority, and the quality of available evidence can outweigh either axis.

The design below uses an invented appointment-booking service. It is a proposal for writing and testing policy, not a description of an internal system or a claim of measured safety.

Start with the action, not the agent

Suppose a customer asks an assistant to move a haircut appointment. The assistant can interpret the request and suggest available times. That does not mean it should have unrestricted access to the salon's calendar.

The action worth assessing is narrower: move this authenticated customer's appointment to a specific slot with their consent, while preserving the service and price. Changing the stylist, adding a cancellation charge, or moving someone else's appointment would be different actions.

That wording exposes decisions a generic "booking agent" permission would conceal. Is the request sufficiently specific to establish consent? Does "same time next week" identify one slot in the customer's local time? Has the customer agreed to a different price? A plausible interpretation is not permission to resolve every ambiguity in the business's favour.

NIST's AI RMF asks organisations to define intended purposes and context in MAP 1.1, then specific supported tasks in MAP 2.1 [1]. The action description applies that principle. It still leaves the service owner responsible for the actual policy.

Replace adjectives with conditions

For this hypothetical service, a candidate unattended rule might require all of the following:

• The requester is authenticated and owns the existing appointment.
• The requested destination is unambiguous and the customer has consented to the exact change.
• The service, stylist, and price remain unchanged, with no cancellation penalty.
• The booking service can reserve the destination and release the original as one consistent change.
• The request matches the current appointment version and has not already been applied.

These conditions are deliberately narrower than "low risk and reversible." They can be checked, denied, and tested. They also separate model work from authority: language interpretation may propose a destination, while the booking service validates ownership and availability.

An absent condition should have a defined destination. Ambiguous intent calls for clarification. A requested price change calls for fresh consent under the service's policy. A conflicting appointment version calls for a fresh read. A request to change another customer's booking should be denied, not sent to an ordinary reviewer as though approval could supply missing authority.

If the booking tool cannot make the move consistently, do not pretend a prompt fixes that limitation. Keep the assistant in recommendation mode until the workflow can handle partial failure or provide a different, explicitly assessed process.

Be precise about undoing an action

Moving an appointment back sounds reversible. But the original slot might have been taken, the stylist might have changed their plans, and the customer might already have received a confirmation.

Reversal of a record and repair of the consequences are different jobs.

For each action, ask what can actually be restored, by whom, and under what conditions. A correction message can reduce confusion without erasing it. A refund can compensate for a charge without making the original experience disappear. If recovery depends on a resource remaining available, that is an assumption to test rather than a promise to print on a diagram.

This also changes how failures should be handled. Imagine the booking request times out. The assistant does not know whether the move happened. Repeating it with a new request identifier submits a new operation instead of recovering the result of the first one.

Amazon's Builders' Library describes caller-provided request identifiers for recognising retries, including the need to coordinate recording the identifier with the mutation [2]. Applied here, the booking service should recognise retries of the same intended move. The policy should also specify how to reconcile an uncertain result before attempting a different move.

Idempotency is not rollback. It addresses repeated requests, not whether the original action was wise or authorised. The API's documented guarantees and retention window still matter.

Write the uncertain cases first

The obvious happy path is rarely where policy authors disagree. Start a review with cases close to the proposed boundary.

The customer writes "next Friday," but their account and browser use different time zones. Two people try to book the final slot at once. The price changes after the assistant presents the options. The customer cancels while a move is awaiting execution. A notification fails after the calendar update succeeds.

For each case, write the expected outcome before running the agent: clarify, deny, refresh the proposal, or report a completed booking with a notification problem. Otherwise, a persuasive explanation after the test can become the policy by accident.

Include near-misses where a single changed fact alters the decision. A test suite containing only clearly allowed and clearly forbidden requests says little about ambiguous intent or timing conflicts.

Model confidence can help decide what information to gather if its use has been evaluated. It should not override ownership, consent, or price constraints. An exceptionally confident interpretation of "whatever works" still does not identify which tradeoffs the customer accepted.

NIST MEASURE 2.3 calls for evaluation under conditions similar to deployment; MEASURE 2.5 calls for documenting limits on generalisation [1]. Neither provision supplies a universal test count that earns autonomy. The evidence needs to address the conditions this particular permission depends on.

Budget for refusal and delay

A stricter boundary can reject requests that would have worked. A looser one can admit changes the customer did not intend. Choosing between them requires an account of both costs.

For the salon, repeated clarification may be annoying enough that customers abandon the booking. But guessing a date can leave a customer without the appointment they expected. Measure unnecessary handoffs separately from unauthorised or incorrect changes. Combining them into one "successful automation" score conceals the tradeoff.

Human review has latency and error costs too. Give the reviewer the exact proposed change, disputed facts, and relevant policy. Do not make them reconstruct the customer's intent from an agent's summary alone. If nobody responds before the slot or proposal expires, preserve the existing appointment rather than treating silence as approval.

Some services may choose a different default because delay itself creates harm. That needs its own authorised policy and evidence. It should not emerge from a timeout handler that quietly proceeds.

Keep the boundary definition versioned with its tests. When a new feature changes cancellation rules or permits a different service, rerun the relevant cases and reconsider the permission. A harmless-looking product change can invalidate the condition on which unattended execution depended.

Before implementing the boundary, ask reviewers to apply it independently to the same awkward cases. Disagreement points to a policy decision still to make; agreement is a useful clarity check, not proof of safety. Better to discover an unresolved question in a test than after the customer's appointment moves.