FBG Reports Support Improvements From a Multi-Agent System

Published · Updated • news
Amazon BedrockMulti-Agent SystemsRAGAWS

The AWS/FBG case study reports approximately 56 percent improvement in containment and 53 percent in resolution during the first two months, based on FBG's internal metrics. Its result statement does not supply absolute baseline rates or isolate the contribution of each component. AI Mastery has not independently reproduced those results.

Correction, 22 October 2026: The previous title called the containment result a reduction in a gap, which the source does not establish. The text also implied model-independent safety and guaranteed throughput from regional routing. Those descriptions are withdrawn. The architecture below is the authors' account, with its deployment conditions retained.

Orchestrator-Plus-Specialist Topology

FBG describes a Supervisor Agent using Claude Sonnet on Bedrock to choose tools and synthesize a reply. Incoming messages pass through Bedrock Guardrails and a Responsible Gaming classifier before the supervisor. Separating those controls from the supervisor does not guarantee a correct safety decision: the classifier is itself model-based, and false negatives, false positives and failed escalation paths still need testing.

The reported request path is FBG mobile app → Salesforce Einstein → Spring AI on Amazon EKS for token validation → Bedrock Guardrails → Responsible Gaming classifier → Supervisor Agent → tools → response. MCP servers hosted as Kubernetes services expose account and transaction queries against REST APIs. RAG retrieval and human escalation are local tools within Spring AI. FBG chose Spring AI for its Java expertise; the case study points to Strands Agents as a Python option with similar orchestration and MCP support, not as a feature-for-feature equivalent verified here.

State-Specific RAG and the Dual-Search Pattern

The RAG pipeline is the system's highest-frequency tool. FBG built it custom rather than using a managed knowledge base to retain precise control over chunking and retrieval. The ingestion pipeline collects state-specific payment method guides, FAQ articles, responsible gaming resources, and account management guides, then splits them using a token-based chunking strategy before embedding with Amazon Titan V2. Vectors are stored in MongoDB Atlas.

At query time, the reported system rewrites the customer's question using an LLM, then combines state-specific and general similarity-search results for jurisdictional questions. The source does not establish that those searches execute concurrently. Combining results supplies evidence to the supervisor; it does not guarantee the final answer applies the correct jurisdiction. The authors report hundreds of new documents monthly, guided by conversation analysis.

Responsible Gaming Classification and Model Sizing

The reported Responsible Gaming classifier uses Amazon Nova 2 Lite with the current message and conversation history. In the described workflow, high-severity classifications trigger human transfer with context, while lower-severity flags are recorded for compliance review. That is the intended routing behavior, not an independently measured detection or transfer success rate.

The authors describe using smaller models where they meet task requirements and reserving Claude Sonnet for supervision. A shared Bedrock API simplifies access, but a model change still needs checks for regional availability, request compatibility, quotas and evaluation outcomes. The case study is not a controlled demonstration that a larger classifier could never improve accuracy.

Model and Service Decisions at Each Layer

Layer Model / Service Rationale
Responsible Gaming classification Amazon Nova 2 Lite Fast, cost-effective; task is well-defined with clear examples and limited outcome set
Supervisor / orchestration Anthropic Claude Sonnet on Amazon Bedrock Complex reasoning, tool orchestration, open-ended natural language generation
RAG embeddings Amazon Titan V2 High-quality vector representations for similarity search in MongoDB Atlas
Prompt injection / content safety Amazon Bedrock Guardrails Input control before the supervisor; detection and escalation require evaluation
Agent and MCP server hosting Amazon EKS Independent autoscaling per service; Kubernetes-native deployment of new tool servers

The authors describe round-robin supervisor routing across Bedrock Regions and independent EKS scaling for the MCP and Spring AI services. Their reported event load can exceed 40 inquiries per two minutes. These mechanisms are not unlimited-capacity guarantees. A proposed load test should include regional throttling, unavailable tools and scaling delays, and check whether the customer receives the correct escalation or failure response.

Evaluation and Iteration

FBG launched with just 4 of their more than 20 support case types, building evaluation infrastructure before expanding scope. An LLM-as-a-Judge pipeline automatically classifies every completed conversation as resolved or unresolved; an operations team reviews those judgments daily and files improvement tickets, creating a compounding feedback loop.

The team reports refining prompts before upgrading models and replaying customer conversations to evaluate changes. Our prompt-optimization discussion describes keeping evaluation criteria stable. An LLM judge's resolution label should be checked against the actual customer outcome and escalation record rather than treated as an independent guarantee.

A proposed evaluation of a similar agentic system should check account identity, retrieval jurisdiction and human transfer separately, then test their interaction. Modularity makes those checks easier to organize; the case study does not establish that the components can be made correct independently or that specialization always justifies its complexity.

Frequently asked questions

Does the case study establish a 56-percentage-point containment gain?

No. It reports approximately 56 percent improvement based on internal metrics, without supplying absolute baseline rates in that result statement.

Do a classifier and guardrails guarantee correct safety decisions?

No. They are controls whose classification and escalation behavior still need evaluation. The classifier is itself model-based.

Does routing across Regions guarantee unlimited throughput?

No. The case study describes FBG's regional routing strategy. A deployment still needs regional model availability, quota and failure-path checks.

Free interactive tools for the decisions this piece raises.

Related Reading