DoorDash Multi-Agent LLM Cleans 50 Flags at $4.79 Each

DoorDash has deployed a production multi-agent LLM system that generated usable pull requests for 45 of 50 evaluated stale feature flags, at an average cost of $4.79 and 13.8 minutes per cleanup. The engineering context makes this significant: the company's experimentation platform spans more than 60,000 flags across roughly 623 repositories, adds approximately 2,300 new flags each month, and had accumulated more than 1,000 stale flags at the time of the project. Manual cleanup was estimated at one to two hours per flag.

The core technical challenge is DoorDash's dependency-injection patterns. A single Boolean flag can scatter its definition, client call, and business logic across five to 20 files including tests, with semantic rather than syntactic relationships between components. Uber's open-source Piranha — which uses abstract syntax tree transformations — didn't fit: AST matching cannot resolve relationships expressed through dependency injection indirection, making LLM-based semantic reasoning the necessary tool for that class of refactoring.

Two-Phase Orchestration Over MCP

The system is built on Google's Agent Development Kit and uses a strict two-phase architecture. In phase one, an orchestrator running Claude Sonnet pulls stale flag tickets from Jira, searches the relevant repositories, and queries the experimentation platform via Model Context Protocol to retrieve flag metadata — rollout percentage, target value, and lifecycle state. A flag is classified as stale when it has gone unmodified for 90 days, remains referenced in source, and is not archived, retired, or explicitly excluded. A daily job creates Jira tickets for qualifying flags.

Before any code changes begin, an engineer reviews the orchestrator's generated report and confirms the target value. That human checkpoint is architecturally deliberate: it gates the destructive phase on verified intent rather than automated inference. This mirrors the infrastructure governance patterns increasingly treated as prerequisites for safe agent deployment, where the boundary between autonomous action and human confirmation is an explicit design decision.

Phase two dispatches Claude Opus cleanup agents into isolated Git worktrees — up to four concurrent agents per repository. Each agent locates flag references, selects a cleanup strategy, modifies source and tests, then runs a full validation chain: build, tests, JaCoCo patch coverage, and Detekt static analysis. A pull request is opened only after all checks pass. Each agent carries a one-hour timeout, and Gradle runs without its daemon specifically to prevent shared state from leaking between worktrees — daemon reuse across isolated worktrees would silently corrupt the isolation guarantee the architecture depends on.

Results by Complexity

The 50-flag evaluation produced 31 first-pass merges requiring no revision, 14 that needed revision, and 5 requiring direct engineer intervention. DoorDash reported zero bugs or regressions across all 50 changes.

The five engineer interventions all involved deep call chains and cross-interface parameter threading — cases where dependency-injected relationships span multiple abstraction layers and the model must hold more concurrent state than it can reliably track.

AI Mastery Analysis

The architectural bet here is that MCP-mediated access to live experiment metadata is what separates this system from a code-rewriting script. Without authoritative rollout percentage and target value at the moment of cleanup, an agent cannot safely determine which branch of a flag's logic to preserve. Grounding phase-one analysis in live platform state — rather than inferring intent from code comments or variable names — is the design decision that makes the zero-regression result credible.

The worktree isolation strategy also deserves attention as a template. Running up to four concurrent Claude Opus agents per repository with daemon-free Gradle builds is a pragmatic answer to parallelism without shared-state corruption, a problem that becomes acute when agents modify overlapping dependency graphs. The cost profile — $4.79 per flag against one to two hours of senior engineer time — is favourable at current complexity levels, but shifts as average complexity increases toward the deep-call-chain cases that required human intervention.

The planned addition of confidence scoring for lower-risk cleanups suggests DoorDash sees the current human checkpoint as a bottleneck worth narrowing. That trajectory — from mandatory human confirmation toward selective autonomy — is consistent with production AI architectures that fail most often on control-plane design rather than model capability. The work has been accepted for the ICSME 2026 industry track, which will surface the formal methodology to peer scrutiny. At 60,000 flags and 2,300 added per month, automated cleanup is not a one-time project but an ongoing infrastructure obligation.

Primary source

DoorDash Uses Multi Agent LLMs to Clean up 60,000 Feature Flags — InfoQ