Build Stateful Web Research Agents with Claude Agent SDK

Research tasks that look simple from the outside often hide a compounding decision problem: the first search result determines the second query, and no predefined workflow covers every branch. If you can only know what to search for after you have read something, a fixed pipeline of API calls will either stop too early or chase the wrong thread. The Claude Agent SDK lets you describe an information need and hand over the search-and-synthesize loop to the model itself.

Who needs this: analysts doing competitive intelligence, engineers investigating failures across documentation and forums, legal teams tracking regulatory changes, and anyone running literature reviews. It costs no local GPU; any Python 3.11+ machine with internet access works. The real cost is API spend. The source notebook uses claude-opus-4-6 as the model, and every web search or image read consumes tokens. An unconstrained research session can loop through many searches before answering, so treat each agent turn as a paid inference step rather than a free function call.

This guide is adapted from Anthropic Cookbook's 00_The_one_liner_research_agent.ipynb, available at https://github.com/anthropics/anthropic-cookbook under the MIT licence. Code is reproduced exactly from that notebook; the prose around it is original.

Prerequisites

You need Python 3.11 or higher, an Anthropic API key, and enough familiarity with async/await to read the control flow. Put the key in a .env file as ANTHROPIC_API_KEY. The notebook assumes you can install Python packages and run async code. Hardware is not a bottleneck because inference happens in Anthropic's cloud.

Install the SDK and environment helper:

%%capture
%pip install -U claude-agent-sdk python-dotenv

Load environment variables and choose a model:

from dotenv import load_dotenv

load_dotenv()

MODEL = "claude-opus-4-6"

claude-opus-4-6 is the source notebook's model. It is high quality but expensive. For a cost-sensitive prototype, you can set MODEL to a cheaper Claude model; the SDK interface stays the same.

Step 1: Build a stateless one-liner research agent

The simplest useful agent is a single query() call. query() creates a stateless interaction: no conversation memory is retained between calls. The async for loop drains messages as Claude plans, invokes WebSearch, and assembles a final answer. allowed_tools=["WebSearch"] grants autonomous web search without asking for human approval.

from utils.agent_visualizer import (
    display_agent_response,
    print_activity,
)

from claude_agent_sdk import ClaudeAgentOptions, query

messages = []
async for msg in query(
    prompt="Research the latest trends in AI agents and give me a brief summary and relevant citiations links.",
    options=ClaudeAgentOptions(model=MODEL, allowed_tools=["WebSearch"]),
):
    print_activity(msg)
    messages.append(msg)

After the loop, render the final response:

display_agent_response(messages)

The source notebook also includes a transcript view:

from utils.agent_visualizer import visualize_conversation

visualize_conversation(messages)

print_activity shows tool calls and reasoning in real time. display_agent_response wraps the final answer as an HTML card, while visualize_conversation renders a search trail. The prompt contains a typo — “citiations” instead of “citations” — in the source notebook. It does not break the demo, but avoid copying that typo into production prompts.

Step 2: Add state, citations, and multimodal input

The one-liner is useful for independent questions, but it cannot do iterative research. A follow-up question starts cold. ClaudeSDKClient fixes this by maintaining conversation state inside a context manager. A system prompt can encode research standards — in this case, always include source URLs, format them as Markdown links, and group them in a Sources: section. Adding the Read tool lets Claude parse images, PDFs, and other files.

from claude_agent_sdk import ClaudeSDKClient

# System prompt with citation requirements for research quality
RESEARCH_SYSTEM_PROMPT = """You are a research agent specialized in AI.

When providing research findings:
- Always include source URLs as citations
- Format citations as markdown links: [Source Title](URL)
- Group sources in a "Sources:" section at the end of your response"""

messages = []
async with ClaudeSDKClient(
    options=ClaudeAgentOptions(
        model=MODEL,
        cwd="research_agent",
        system_prompt=RESEARCH_SYSTEM_PROMPT,
        allowed_tools=["WebSearch", "Read"],
        max_buffer_size=10 * 1024 * 1024,  # Increase to 10MB for image handling
    )
) as research_agent:
    # First query: Analyze the chart image
    await research_agent.query("Analyze the chart in research_agent/projects_claude.png")
    async for msg in research_agent.receive_response():
        print_activity(msg)
        messages.append(msg)

    # Second query: Use web search to validate/contextualize the chart findings
    await research_agent.query(
        "Based on the chart analysis, search for recent news or data that validates or provides context for these findings. Include source URLs."
    )
    async for msg in research_agent.receive_response():
        print_activity(msg)
        messages.append(msg)

Then view the transcript if you want a complete audit of the two-turn session:

visualize_conversation(messages)

The first query uses Read to inspect a chart. The second query runs in the same conversation, so Claude can refer back to the chart while searching the web. Conversation state is preserved across receive_response() calls inside the async with block.

Large base64-encoded images or tool outputs can exceed the SDK buffer. The notebook sets max_buffer_size to 10MB for image handling. If you work with very large PDFs or many images, raise it further and watch your memory use. The cwd="research_agent" directory must exist and contain the chart; otherwise the Read tool fails.

Step 3: Use the production send_query() wrapper

For reusable code, the source notebook packages the same logic into research_agent/agent.py. send_query() handles activity display, context reset, and styled result rendering internally. Each call without continue_conversation starts a fresh conversation; with True, it continues the previous one.

from research_agent.agent import send_query

# The module handles activity display, context reset, and result visualization internally
result = await send_query("What is the Claude Code SDK? Only do one websearch and be concise")

Run a second independent question:

result1 = await send_query("What is Anthropic? Only do one websearch and be concise")

Continue the second conversation to dig deeper:

# Continue the conversation to dig deeper by setting continue_conversation=True
result2 = await send_query(
    "What are some of their products?",
    continue_conversation=True,
)

The prompt constraints — “Only do one websearch and be concise” — are not decorative. They cap the number of tool calls and keep cost predictable. Without those constraints, an autonomous agent may decide to run several searches before answering.

Step 4: Verify outputs with outcome-based evaluation

Outcome-based evaluation means judging the agent not by whether it ran tools, but by whether the answer satisfies pre-registered outcomes. For each research request, define checks before the run: does the answer directly address the question? Are there at least N distinct sources? Do the cited URLs resolve? Does each source actually support the claim it is attached to? A citation that exists but contradicts the answer is worse than no citation.

Because send_query() returns the result object in the notebook, you can wire these checks into a script or review them manually in a notebook. If you need programmatic access without the styled HTML card, inspect research_agent/agent.py and use the underlying return value instead of the display wrapper. For teams that need reproducible evaluations, offline checks avoid contamination drift — Anthropic's shift toward offline evals explains why pinning a held-out question set matters more than live scoring.

What to watch out for

• Buffer overflows are silent until they are not. Large base64 images and tool outputs can exceed the SDK buffer. Raise max_buffer_size before you need it, and keep an eye on context length because larger buffers mean larger potential token bills.
• Stateless calls do not remember anything. If you ask “What is Anthropic?” and then “What are their products?” without continue_conversation=True, the second call starts fresh. This is a feature for parallel research, but a bug for follow-ups.
• Unconstrained web search gets expensive. The source examples include “Only do one websearch and be concise” for a reason. Without bounds, an agent can loop through searching, reading, and synthesis many times before answering. If you are cost-sensitive, keep those constraints or add your own maximum-turn guard.
• Citations are not verification. The system prompt ensures citation formatting, not factual accuracy. A URL can look plausible and still be hallucinated. Validate source domains and content before trusting reports.
• cwd="research_agent" must exist and contain the chart. If the directory or projects_claude.png is missing, the Read tool fails. Set the working directory explicitly in your own scripts.
• Model choice dominates cost. claude-opus-4-6 is the notebook default. For high-volume research jobs, benchmark a cheaper Claude model against the same outcome checks before scaling.

Where to go next

This foundation covers single-agent research. The natural next step is multi-agent orchestration: a coordinating agent that dispatches specialized subagents, adds governance through hooks and custom commands, and formats outputs for different stakeholders. Later notebooks in the Anthropic Cookbook build toward that pattern. For deeper prompting patterns for research agents, see the patterns/agents/prompts directory in the same repository. If you move this into production, treat the evaluation step as part of the agent, not an afterthought.