Self-Verification for AI Agents: Build an Evaluator-Optimizer Loop in LangGraph
In this article
The code in this guide is reproduced from the Evaluator-optimizer section of LangChain's official LangGraph workflows and agents documentation (langchain-ai/docs, MIT licence). The import block is ours, labelled as such, because the documentation defines its imports elsewhere on the page.
The hardest problem in autonomous agents is not getting one step right. It is that steps compound. If every step of a 20-step task succeeds 95% of the time, and failures are independent, the whole task succeeds only about 36% of the time (0.95²⁰ ≈ 0.358). And once step 3 goes wrong, steps 4 to 20 build on the mistake.
Self-verification breaks that chain: check each output against a success criterion before anything builds on it, and send failures back with specific feedback. LangGraph calls the pattern evaluator-optimizer: one LLM call creates a response, another evaluates it, and the loop continues until the response is acceptable. This guide builds it from LangGraph's documentation, then turns the toy example into something you can trust in an agent: a bounded loop, a structured verdict, and evaluators that check rather than opine.
Setup
The documentation uses Anthropic's chat model; any model that supports structured output works:
pip install langchain_core langchain-anthropic langgraph
import os
import getpass
from langchain_anthropic import ChatAnthropic
def _set_env(var: str):
if not os.environ.get(var):
os.environ[var] = getpass.getpass(f"{var}: ")
_set_env("ANTHROPIC_API_KEY")
llm = ChatAnthropic(model="claude-sonnet-4-6")
The workflow code below uses names the documentation imports elsewhere on its page. This import block is ours:
from typing import Literal
from typing_extensions import TypedDict
from pydantic import BaseModel, Field
from langgraph.graph import StateGraph, START, END
Step 1: Define the state and the verdict
The graph's state carries the task, the latest attempt, and the evaluator's verdict between nodes. The evaluator answers through a schema, not free text:
# Graph state
class State(TypedDict):
joke: str
topic: str
feedback: str
funny_or_not: str
# Schema for structured output to use in evaluation
class Feedback(BaseModel):
grade: Literal["funny", "not funny"] = Field(
description="Decide if the joke is funny or not.",
)
feedback: str = Field(
description="If the joke is not funny, provide feedback on how to improve it.",
)
# Augment the LLM with schema for structured output
evaluator = llm.with_structured_output(Feedback)
The documentation's task is jokes, but the shape is what matters. grade is a Literal with exactly two values, so the routing decision is a fixed choice the code can branch on, not a sentence to interpret. feedback is required, so a rejection always comes with something the generator can act on. Swap the joke for a SQL query, a summary or a patch, and keep this shape.
Step 2: The generator and the evaluator
# Nodes
def llm_call_generator(state: State):
"""LLM generates a joke"""
if state.get("feedback"):
msg = llm.invoke(
f"Write a joke about {state['topic']} but take into account the feedback: {state['feedback']}"
)
else:
msg = llm.invoke(f"Write a joke about {state['topic']}")
return {"joke": msg.content}
def llm_call_evaluator(state: State):
"""LLM evaluates the joke"""
grade = evaluator.invoke(f"Grade the joke {state['joke']}")
return {"funny_or_not": grade.grade, "feedback": grade.feedback}
On the first pass there is no feedback, so the generator works from the task alone. On every later pass it receives the evaluator's specific criticism, which is what makes the loop converge instead of producing random retries. Each node returns only the keys it changes, and LangGraph merges them into the state.
Step 3: Route on the verdict and build the graph
# Conditional edge function to route back to joke generator or end based upon feedback from the evaluator
def route_joke(state: State):
"""Route back to joke generator or end based upon feedback from the evaluator"""
if state["funny_or_not"] == "funny":
return "Accepted"
elif state["funny_or_not"] == "not funny":
return "Rejected + Feedback"
# Build workflow
optimizer_builder = StateGraph(State)
# Add the nodes
optimizer_builder.add_node("llm_call_generator", llm_call_generator)
optimizer_builder.add_node("llm_call_evaluator", llm_call_evaluator)
# Add edges to connect nodes
optimizer_builder.add_edge(START, "llm_call_generator")
optimizer_builder.add_edge("llm_call_generator", "llm_call_evaluator")
optimizer_builder.add_conditional_edges(
"llm_call_evaluator",
route_joke,
{ # Name returned by route_joke : Name of next node to visit
"Accepted": END,
"Rejected + Feedback": "llm_call_generator",
},
)
# Compile the workflow
optimizer_workflow = optimizer_builder.compile()
# Show the workflow
display(Image(optimizer_workflow.get_graph().draw_mermaid_png()))
# Invoke
state = optimizer_workflow.invoke({"topic": "Cats"})
print(state["joke"])
The conditional edge is the self-verification: after every evaluation, an accepted result goes to END and a rejected one goes back to the generator with feedback. The display(Image(...)) line renders the graph in a notebook; outside one, drop it or import display and Image from IPython.display.
The same loop with the Functional API
LangGraph's Functional API expresses the same workflow as an ordinary loop:
# Nodes
@task
def llm_call_generator(topic: str, feedback: Feedback):
"""LLM generates a joke"""
if feedback:
msg = llm.invoke(
f"Write a joke about {topic} but take into account the feedback: {feedback}"
)
else:
msg = llm.invoke(f"Write a joke about {topic}")
return msg.content
@task
def llm_call_evaluator(joke: str):
"""LLM evaluates the joke"""
feedback = evaluator.invoke(f"Grade the joke {joke}")
return feedback
@entrypoint()
def optimizer_workflow(topic: str):
feedback = None
while True:
joke = llm_call_generator(topic, feedback).result()
feedback = llm_call_evaluator(joke).result()
if feedback.grade == "funny":
break
return joke
@task and @entrypoint come from langgraph.func. This version is easier to read, and it makes the main risk visible: while True has no exit except approval.
Step 4: Make it safe to run unattended
The documentation's example is deliberately minimal. Before an agent depends on it, add four things:
- A hard cap on rounds. In the Graph API, pass a limit when you invoke, for example
optimizer_workflow.invoke({"topic": "Cats"}, {"recursion_limit": 10}), and LangGraph raises an error instead of cycling indefinitely. Better, keep a round counter in the state and route toENDwith the best attempt when it is reached. In the Functional API, replacewhile Truewith a boundedforloop. - Mechanical checks before an LLM judge. If the output is code, run its tests. If it is JSON, validate it against a schema. If it is SQL, run
EXPLAINagainst a read-only replica. These checks cannot be talked round, and they cost almost nothing. Use an LLM evaluator only for what code cannot judge. - A strict, specific evaluator prompt. "Grade the joke" works for jokes. For real outputs, state the criteria the evaluator must check, in order, and require feedback that names the failing criterion.
- A different evaluator from the generator. The same model tends to approve its own mistakes. Use a different model, or at minimum a separate, stricter prompt with structured output.
| Evaluator type | Good for | Weakness |
|---|---|---|
| Deterministic checks (tests, schema validation, linters) | Code, structured data, anything with a correct answer | Only checks what you encode |
| LLM judge with a fixed verdict schema | Tone, completeness, faithfulness to a source | Can be confidently wrong; shares blind spots with the generator if they are the same model |
| Human in the loop | High-stakes or irreversible actions | Slow and expensive; use LangGraph interrupts only where it matters |
What this looks like in production
We run this exact pattern to publish this site: one model drafts each article and a different model fact-checks and rewrites it against the source before a gate decides whether it publishes. When we planted 20 wrong figures in drafts to test four LLM editors, the lessons matched the list above. The evaluators' self-reported confidence was nearly constant and useless as a gate; their structured lists of unverified claims, plus mechanical checks for length, structure and links, did the real work. Even the best editor let one planted error in twenty through, which is why the mechanical checks stay in place.
Where to go next
For verifying factual claims specifically, Chain-of-Verification with SGLang breaks an answer into questions and checks each one independently. To see an evaluator loop inside a retrieval agent, build the self-correcting RAG agent with LangGraph and Milvus. And for designing LLM judges that hold up, read mastering advanced RAG evaluation.
Frequently asked questions
What is an evaluator-optimizer workflow?
One LLM call produces a response and a second evaluates it against a success criterion. If the evaluator rejects it, its feedback goes back to the generator, which tries again; the loop ends when the evaluator accepts. LangGraph documents it as a standard workflow pattern alongside prompt chaining, routing and orchestrator-worker.
Why should an AI agent verify its own steps?
Errors compound across steps. If each step of a 20-step task is 95% reliable and failures are independent, the chance that every step succeeds is about 36% (0.95 to the power 20). Checking each step before moving on stops one early mistake from corrupting everything after it.
How do you stop an evaluator-optimizer loop from running forever?
Cap it. In LangGraph's Graph API, pass a recursion_limit in the invoke config so the graph raises an error instead of cycling indefinitely, and keep a round counter in state so you can return the best attempt. The Functional API example uses while True, so add an explicit attempt counter there.
Should the evaluator be an LLM or code?
Use code wherever the criterion can be checked mechanically: tests for generated code, schema validation for JSON, a compiler or linter, a SQL dry run. Reserve an LLM evaluator for criteria code cannot judge, and give it a structured verdict such as a fixed grade plus feedback instead of a free-text opinion.
Can the same model generate and evaluate?
It can, but it tends to share its own blind spots. A different model, or at least a differently prompted one with a strict structured output, catches more. In our own benchmark of LLM editors, self-reported confidence scores were nearly constant and useless as a gate, while structured lists of unverified claims carried the decision.
Related Guides
Build Stateful Web Research Agents with Claude Agent SDK
Run one-off and stateful Claude research agents that search the web, cite sources, and read multimodal inputs with the Anthropic Cookbook pattern.
NOOA: NVIDIA's Object-Oriented Agent Framework Explained
NVIDIA open-sources NOOA, a Python framework that collapses prompt templates, tool schemas, and workflow graphs into one class—with 82.2% on SWE-bench Verified.

RAG vs MCP in 2026: What Each Solves and How They Fit Together
RAG is a technique for grounding answers in your data; MCP is a protocol for connecting models to tools and data. Build an MCP server and see where RAG fits.