Silent failures: when an agent replies but does not solve the task

On 24 July 2026, AWS described a problem that availability metrics miss: an agent can answer without a technical error and still fail the task. It may query the wrong source, repeat a call, stop before acting, or claim an action that the destination system does not confirm. If a team measures only latency, cost and error codes, those cases look successful.\n\n## Measure the goal, not only the reply\n\nAgentCore Insights documentation analyses session traces and groups failures by cause. It includes tool errors, hallucinations, incorrect actions, context problems, repetition and premature termination. The useful point does not depend on AWS: before deployment, define which verifiable outcome counts as success. “It replied” is not enough; “the appointment was created”, “the report contains the correct source” or “the order has confirmation” can be checked.\n\n## Follow the path\n\nA session retains conversation context; a trace shows an interaction; a span records an operation such as a tool call. That hierarchy, explained by AWS, lets a team move from a complaint to the decision or call that changed the outcome. Retain the goal, tools, inputs, outputs, reply and destination-system confirmation. Always filter sensitive data.\n\n## Five checks\n\nFirst, review a sample of sessions marked successful and verify the effect outside the agent text. Second, label the cause: data, permission, instruction, tool or decision. Third, group repeated causes. Fourth, change one thing —a description, permission or validation. Fifth, compare the variant against equivalent cases before calling it better. Automatic evaluators help, but can also be wrong; keep human-reviewed examples.\n\nA reliable agent is not one that never throws an exception. It leaves enough evidence to find a fluent answer that did not solve the work. The question worth automating is: what proves that the user goal was completed?