Muse Voice Transcribe: 3.1% WER, One Model for ASR, Diarization, Endpointing

Meta Superintelligence Labs this week shipped Muse Voice Transcribe, an autoregressive multimodal model that collapses streaming automatic speech recognition, speaker diarization, and endpointing into a single decoder pass. Traditional production voice stacks chain three separate systems — a transcription model, a speaker-separation model, and a voice-activity detector — and every hand-off between them adds latency and a new failure mode. Muse Voice Transcribe eliminates those boundaries, processing audio with no required post-processing step. For engineers building real-time AI voice agents, where compounding pipeline latency is often the binding constraint on responsiveness, the architectural decision matters as much as any accuracy figure.

Single Decoder, Three Tasks

The model belongs to the Muse Spark family and operates on 80 ms audio chunks ingested at 12.5 Hz. Each chunk is encoded into one soft token. After every chunk the decoder makes a binary choice: emit a <|next_audio|> token and consume the next chunk, or emit a text token and write a word. When the audio stream closes, an <|empty_audio|> token flushes all remaining text without requesting further input. Listening and writing share one decoder loop, so there is no separate alignment stage that can drift under load.

Diarization and endpointing are not bolted-on sub-models — they are additional token types trained jointly with the ASR objective. A <|start_of_turn|> token fires the moment a speaker switch is detectable; the corresponding speaker tag is delayed to the end of the current chunk. That asymmetry lets the model attribute contiguous segments from the same speaker to one tag even when separated in the stream. Endpointing follows the same pattern: <|speech_onset|> marks utterance start and <|speech_endpoint|> marks completion, removing the need for a standalone voice-activity detector. This is consistent with the broader shift described in architectural specificity outperforming GPU scaling alone — gains here come from token-level design, not model size.

Reinforcement-Learned Adaptive Delay

Because the model decides when to listen, it also controls how much audio context accumulates behind each word — a gap Meta labels "delay." A naive fixed delay forces a blanket trade-off between accuracy and latency. Instead, Meta trains the policy with reinforcement learning that combines a word error rate reward and a delay reward multiplicatively, so the model learns to hold longer on acoustically difficult words and commit faster on clear ones. Meta reports this places Muse Voice Transcribe on the Pareto front for speed versus accuracy, ahead of the prior frontier formed by Soniox, Cartesia, and ElevenLabs systems.

Benchmark Performance and Pricing

On Artificial Analysis AA-WER Streaming, measured as of September 1, 2026, Muse Voice Transcribe records 3.1% final-transcript WER at 0.16 s after end of speech, and 3.6% WER on first partial transcript at 0.13 s. On diarization, Meta reports a 17.5% average diarization error rate across AMI-IHM, AMI-SDM, and VoxConverse; five comparison systems in the same evaluation range from 21.1% to 28.6%.

At $3.00 per 1,000 minutes ($0.18 per audio hour), Muse Voice Transcribe undercuts Cartesia Ink-2 by 25% and costs less than half the per-minute rate of both ElevenLabs Scribe v2 Realtime and Deepgram Flux. The Cartesia Ink-2 configuration with external endpoints is faster at 0.07 s but records the worst WER of the group at 4.0% — precisely the speed-accuracy trade the adaptive delay policy is designed to avoid.

Availability and Constraints

The model is live on the Meta Model API as muse-voice-transcribe-1.0 and already powers dictation in Meta AI for Mac and Muse Code. Training covered 70+ languages, with 25 extensively verified and recommended at launch. Native code-switching operates both within and across sentences, and accuracy can be improved through language, keyword, and context biasing. Audio exceeding one hour with 20+ simultaneous speakers is supported without post-processing.

The firm constraint is deployment model: no weights have been released, so teams with data-residency requirements or API round-trip latency budgets they cannot absorb are blocked out. This API-only posture aligns with Meta's recent efficiency-driven restructuring. For the broader agent ecosystem — where pipeline architecture rather than raw model quality now drives production gains — the more significant takeaway is that collapsing three failure-prone pipeline stages into one token stream is the pattern to watch. Competitors still operating three-system stacks face an architectural disadvantage that cannot be closed by swapping in a better transcription model alone.