Five Ways to Question an AI Agent's Success

"The tool returned success" is a statement about a tool call. It leaves open whether the requested outcome happened, whether it lasted, and what else changed.

That gap is not unique to AI. Google's SRE book describes an HTTP success response containing the wrong content as an implicit error. It distinguishes monitoring internal behaviour from testing what a user can actually observe [1]. An agent that writes a convincing completion summary adds another place where a success claim can become detached from the outcome.

I use five questions to examine that gap: was the explanation wrong, was the scope wrong, did the result last, did harm appear elsewhere, and was the context still current? This is a proposed review framework. It is not an exhaustive taxonomy or a measured ranking of agent failures. The categories overlap, and none is inherently invisible to an automated check.

The examples below describe an invented online shop. They are independent illustrations, not production incidents.

A plausible explanation for the wrong problem

The shop's agent sees that a product is missing from search. It concludes that the search index is stale and requests a rebuild. The rebuild finishes. The product remains hidden because its publication status was never enabled.

Everything about the action can be technically correct while the diagnosis is wrong.

The useful check tests an observation that distinguishes the competing explanations. Before rebuilding, inspect whether the product is eligible for publication. Afterwards, query the customer-facing search path. Another fluent explanation of why rebuilding should work does not supply that evidence.

A second model may help propose alternatives, but it may also repeat the same assumption. Its agreement is not an independent measurement of the shop. Likewise, a model's stated confidence is not a calibrated probability unless that interpretation has been validated for the task and conditions.

If the evidence cannot distinguish causes, the agent can gather more information or hand over the investigation without making a change. A human reviewer should see the unresolved alternatives rather than just a polished preferred answer.

The right operation on too many objects

Suppose the agent correctly identifies a pricing error in a seasonal collection. It sends a valid price-update request but selects the whole catalogue instead of that collection. The tool can truthfully report that the update succeeded.

The failure is in the target set.

Record the intended set before execution and compare it with the resolved targets. The mutation service should reject targets outside the authorised collection, even if the agent's summary says the scope is narrow. A readable preview helps a reviewer, but the actual object identifiers matter more than the label on the approval button.

Verification should inspect both the corrected prices and whether unrelated products changed. Testing only the selected collection misses the defining error in this example.

Humans can overlook scope too, especially when a preview is long or incomplete. Use enforceable limits as well as review. If the tool cannot expose its resolved targets or constrain the update, that limitation belongs in the decision about whether to allow it.

Improvement that does not last

The agent removes an incorrect "out of stock" label. The product becomes available. A scheduled import later restores the wrong label because the upstream inventory record still contains the error.

An immediate customer-facing check would pass. It would still be useful, but it would not establish that the problem was resolved.

Choose the follow-up condition from the mechanism. In this example, verify the upstream record and check again after the relevant import has run. A universal rule to recheck every task after the same number of hours would be arbitrary. Some outcomes need event-based checks; others need an observation period chosen for the task.

Keep execution complete, immediate outcome verified, and follow-up pending as separate states. If follow-up fails to run, the result remains unverified. It should not quietly become a success because no new alert appeared.

When a problem returns, preserve the link to the earlier action. Otherwise, separate tickets can make repeated temporary relief look like several independent resolutions.

A local improvement with a downstream cost

The shop's agent retries a backlog of order-confirmation emails. The backlog shrinks, but the increased traffic delays password-reset messages using the same delivery service.

Local success and wider harm can coexist. Watching only the confirmation queue would miss the tradeoff.

Define relevant downstream checks before enabling the action. Here, delivery latency for other message classes and the shared service's limits are pertinent. That does not require a complete model of every dependency. It requires an explicit account of which neighbouring effects the team can observe and which remain uncertain.

If delays follow the retry burst, timing is a reason to investigate, not proof of causation. Compare the action record with delivery-service telemetry and other concurrent changes. Automated correlation can identify a lead without settling the diagnosis.

The response also needs care. Blindly undoing or retrying work again can make the situation worse. Pause further submissions where appropriate, inspect what was accepted, and give the incident owner the evidence needed to choose a recovery action.

A sound decision on expired facts

The agent prepares an inventory correction while a warehouse import is still running. By the time the write arrives, the stock record has changed. Applying the earlier correction overwrites newer information.

Freshness is a property of the evidence and the write protocol, not the quality of the explanation.

Capture the version of the record used to prepare the update. Where the storage system supports it, require the write to match that version. A conflict should trigger a fresh read and reassessment, not an automatic repeat of the old write. A recent timestamp alone cannot establish that no intervening change occurred.

If the underlying tool offers no conditional update, the team needs another means of coordination or a narrower permission. Asking a human to review an equally stale snapshot does not repair the race.

This category can cause several others. Stale collection membership can lead to excessive scope; a stale inventory record can lead to the wrong diagnosis. Classifying the event in more than one place is fine if that exposes the missing checks.

Keep the result open to challenge

A useful completion record lets another observer challenge it. Preserve the requested outcome, actual targets, evidence versions, and tool result. Attach the observations supporting the outcome, plus any outstanding follow-up. Restrict access and retention where those records contain sensitive information.

The verifier should obtain evidence beyond the agent's narrative. Customer-visible tests and authoritative state reads can help, but they also have limitations. If the verifier shares the agent's stale cache, its apparent independence adds little. If monitoring is unavailable, report that uncertainty instead of interpreting silence as health.

NIST's AI RMF calls for production monitoring in MEASURE 2.4 and post-deployment plans covering incident response, recovery, and change management in MANAGE 4.1 [2]. It does not prescribe these five categories or establish their frequency.

Use the questions to construct tests before expanding permissions. Deliberately give the test environment a stale record, an overbroad target set, or a delayed reappearance of the fault. Then check whether the system reports the right outcome and whether the escalation reaches someone who can act.

The most useful status may be "the action completed, but we have not yet verified the result." Leave room for it.