Kyutai's 9B Voice Model Hits 77.1% on Spoken GSM8K via RL

Kyutai has released two open-weight checkpoints under the name Voice of Reason — both 9B-parameter models built on GLM-4-Voice-9B that solve spoken mathematics problems without any transcription step, separate text LLM, or text-to-speech post-processing stage. The headline result is a jump in spoken GSM8K accuracy from 27.3% (GLM-4-Voice base) to 77.1% (Voice of Reason Stitch, released checkpoint), achieved through supervised fine-tuning followed by reinforcement learning applied entirely in the audio-interleaved token space.

Architecture and token-stream design

GLM-4-Voice emits output in a fixed interleaved pattern: 13 text tokens, then 26 audio tokens, repeating. Voice of Reason preserves this constraint and releases two variants. glm-4-voice-of-reason-9b answers directly — any reasoning is spoken aloud in that interleaved stream. glm-4-voice-of-reason-stitch-9b inserts silent 100-token reasoning chunks before each spoken block; Kyutai schedules later chunks to generate while earlier speech plays, so the extra reasoning adds no perceptible latency.

Two engineering choices made the RL objective work. First, audio-token merging: at each audio position, all audio-vocabulary probabilities are summed into a single abstract token. The loss asks only whether audio followed, not which audio token appeared. The paper proves this estimator is unbiased and lower-variance under a value-invariance assumption — a result that lets the policy gradient flow through audio steps without the variance explosion that treating each audio token individually would introduce.

Second, temperature correction: the model samples at temperature 0.9 during rollout. Without dividing logits by that temperature inside the log-softmax of the RL loss, the gradient is computed under the wrong distribution. Removing the correction collapsed spoken GSM8K from 65.5% to 12.3% — a finding with direct implications for anyone porting GRPO-style training to mixed-vocabulary models. This is precisely the kind of architectural specificity that outperforms naive GPU scaling in practice.

Training procedure

Stage-1 SFT trained on 150,616 problems from Orca-Math. Qwen3-235B rewrote each problem for spoken delivery; Kyutai's DSM TTS voiced them across multiple speaker identities. SFT alone lifted GLM-4-Voice from 27.3% to 61.7%.

Stage-2 RL sampled 4 replies per spoken prompt at temperature 0.9. Qwen3-235B-A22B-2507 scored each decoded text stream with a binary reward without seeing the reference answer; on 100 human-checked cases it agreed 88% of the time. Rewards were centered within each group — group-relative REINFORCE without PPO clipping or KL regularisation. Training ran for 1,500 updates on 16 H100 GPUs.

Benchmark results

Paper scores use top-k 50 decoding averaged over 3 seeds; released checkpoints drop top-k, yielding the higher figures. Additional findings: gains survive transcription — running Qwen3-ASR-1.7B over the Stitch model's audio output yielded 72.0 ± 1.9% on GSM8K. Speech naturalness (UTMOSv2) held essentially flat after RL. RL shortened the direct model's average reply from 41.9 to 36.4 seconds, ruling out length inflation. Spoken TriviaQA fell from 40.6% to 34.0% for the direct model, though the authors attribute most of this to SFT rather than RL.

AI Mastery analysis

Three things warrant scrutiny before deploying these checkpoints. First, contamination: 54.0% of the 678 GSM8K questions checked overlapped with Orca-Math training data at the paraphrase level, and the team excluded four sub-benchmarks (AddSub, MultiArith, SingleEQ, SVAMP) for this reason. The 77.1% figure should be read with that overlap in mind, even though evaluation audio came from GPT-4o-mini-TTS — a different TTS system than the DSM-voiced training data — and answers were judged by GPT-4o.

Second, the gap between speech-native models and cascaded pipelines remains substantial. A 31B-LLM cascade reaches 95.7% versus 77.1% for Voice of Reason Stitch — an 18.6-point deficit. Cascades pay in latency and lose paralinguistic information, but they still dominate on accuracy. The infrastructure costs of cascaded pipelines are real, yet engineers should not mistake narrowing the gap with closing it.

Third, the RL reward signal is binary and model-judged, with 12% disagreement against human labels. For math, binary correctness is relatively clean; extending this recipe to open-ended spoken tasks will require a more expressive reward model or human-in-the-loop scoring that changes compute economics substantially.

On the positive side, audio-token merging and temperature correction are transferable to any interleaved audio-text architecture. That 1,500 RL updates on 16 H100s produced a 15.8-point lift over STITCH without touching model size is a concrete data point for teams weighing architecture-specific training investments. Both checkpoints run on a single H100 in BF16 and are available on Hugging Face; deployment requires the GLM-4-Voice repository for the speech tokenizer and decoder, and no managed inference provider hosts them yet.

Primary source

Kyutai Releases Voice of Reason: A Speech-Native Model that Solves Spoken Math with Reinforcement Learning — MarkTechPost