Diagnosing Production AI Failures: Errors, Evidence and Side Effects

A production incident can arise from unavailable capacity, missing evidence, an incorrect answer or a wrong action. Start by identifying which event occurred. A plausible final response does not tell you whether the preceding retrieval and tool work was correct.

Correction, 22 October 2026: The earlier article claimed architecture rather than model quality causes production AI failures. It misquoted Datadog's error data as a silent-failure statistic, treated a proposed watchdog threshold as a general solution and described a RAG reference workflow as a proven elimination of hallucination. Those conclusions are withdrawn.

Count the errors the report actually measured

Datadog's report says 5% of observed LLM call spans reported an error in February 2026, with rate limits responsible for 60% of those errors. Its March figures are different: 2% of spans reported errors, with rate limits accounting for almost one-third.

The complement of the rate-limit share is not a measurement of semantically wrong HTTP 200 responses. Nor does call-span telemetry alone establish the error rate of an entire user task. Datadog also identifies its customer population as an imperfect sample of the global market.

Use an operational error metric for operational questions. To measure wrong-but-successful answers, define an outcome evaluator and a labeled sample that includes requests returning normally. Do not manufacture that rate by subtracting unrelated categories.

Inspect the evidence before diagnosing the model

AWS's utility-bill reference workflow describes preprocessing before ingestion into a Bedrock knowledge base. It is a useful implementation example, not a controlled test showing that model changes are ineffective.

There is also an important format distinction. Textract's operation limits list JPEG, PNG, PDF and TIFF. Do not infer direct DOCX or XLSX API support from a workflow's list of source formats. Confirm the parser or conversion stage and test its output.

For each failing question, inspect the original document, extracted representation, retrieved passages and answer. A missing table row and an answer that ignores a correctly retrieved row are different defects. They can require different changes.

Retain an unchanged test set while comparing a parser, retriever, prompt or model modification. Changing several stages together may improve the system, but it does not isolate which stage caused the improvement.

Check the action's effect

In her QCon presentation, Zhou Yu distinguishes an agent saying a product was returned from checking whether the return happened in the backing environment. Her observation that many agents remain demos is not a measured census proving a universal 95% failure rate or a single cause.

A proposed test for a support agent should therefore check the requested record, operation and final state. For example, a well-formed empty account-history result needs to be distinguished from a verified account with no history. Validate identity and provenance before using it to make a customer-facing decision.

Simulate scenarios without treating simulation as certification

ArkSim provides synthetic multi-turn user conversations and evaluations. It can help exercise follow-ups and tool paths that a single-turn answer test does not cover. Its usefulness depends on the scenarios and evaluator you configure.

Use isolated fixtures for mutations. Include cases with ambiguous identifiers, unavailable services, changed permissions and an operation that times out after taking effect. Check whether the next attempt duplicates the action.

This table is our proposed diagnostic procedure. AI Mastery has not run a comparative production study establishing which intervention wins. Our editor-evaluation method covers how to retain the raw evidence needed for a reproducible comparison.