Perplexity JOLT Cuts RL Training Tokens 6.5x on MATH

Reinforcement learning post-training is constrained by sparse rewards: outcome-reward methods such as GRPO assign one reward per trajectory, leaving intermediate tokens without a learning signal. Perplexity and NVIDIA's JOLT (Joint On-policy Learning and Teaching) attacks that sparsity by training a single shared model as both teacher and student. The teacher role sees privileged information; the student role does not.

MarkTechPost reports the headline token figures from the JOLT paper: on MATH with Qwen3-0.6B, JOLT needed 1.30 million processed training tokens to match 63.44% accuracy, versus 8.40 million for GRPO's checkpoint—a 6.5× reduction. Counting only completion tokens widens the gap to roughly 13×. For teams treating token budgets as infrastructure spend, token efficiency is already repricing production AI.

What JOLT changes about the teacher-student relationship

Prior self-distillation work, OPSD, already removed the separate teacher by giving the student privileged context and using its output as a dense token-level training signal for the unprivileged student. Perplexity's paper argues that is insufficient: a teacher with access to the answer can take shortcuts the student cannot replicate, and its guidance may sit beyond the student's current capability. Teacher-student disagreement grows during MATH training, according to the paper.

JOLT adds an explicit teacher-training objective alongside the student's distillation objective. The teacher is updated from its own trajectories with an outcome reward plus a KL penalty that keeps its action distribution close to the student's current policy. The target is a "reward-consistent" teacher whose distillation update points in the same direction as the student's reward gradient up to positive scaling (1/β_T); larger β_T keeps the teacher closer to the student, while smaller values let it favor high-advantage actions more aggressively. JOLT+ adds the student's own outcome reward on top of the distillation signal.

Benchmark results

MarkTechPost reports the research team tested five settings, each using a different base model and form of privileged information. The table below shows four published benchmark settings; a fifth, GSM8K on DeepSeek-V2-Lite, gave JOLT+ a 1.5-point gain over GRPO. All scores are vendor-reported from the JOLT paper, as tabulated by MarkTechPost, with hyperparameters tuned per method.

In the table, JOLT+ holds the highest mean score in every setting. The largest absolute gain is AppWorld: 72.09% state-test pass rate versus GRPO's 62.20%, a 9.9-point margin. On Terminal-Bench 2.1—one training run per method with Qwen3-30B-A3B-Instruct trained on TMax—JOLT+ reaches 10.96% pass@1 and 34.83% pass@32, against GRPO's 8.25% and 30.34%. The unpost-trained base model scores 5.18% pass@1 and 22.81% pass@32.

Plain JOLT without the student outcome reward trails GRPO on AppWorld (51.92% vs 62.20%) and Terminal-Bench pass@1 (6.56% vs 8.25%). Distillation alone is not sufficient on long agentic tasks unless paired with the student's own reward.

AI Mastery analysis

The 6.5× processed-token reduction comes from fitted accuracy curves, not a direct checkpoint comparison: the 63.44% figure uses GRPO's observed checkpoint against JOLT's fitted curve. Training hardware and wall-clock runtime are not disclosed, so token count is a proxy for compute cost rather than a direct measurement. That limits any immediate translation into a training budget, and it mirrors the broader pattern that infrastructure rewrites, not model weights, drive the efficiency gains that matter in production.

AppWorld deserves scrutiny. Plain JOLT's 51.92%—about 10 points below GRPO—with a ±5.30 standard deviation suggests the teacher's distillation signal alone can harm agentic-task performance. JOLT+ recovers and surpasses GRPO, but its dependence on the student outcome reward makes JOLT an augmentation requiring both signal types rather than a clean drop-in replacement. Terminal-Bench's single-run design provides no seed-variance estimate, so confidence in those numbers is lower than for the three-run MATH and AppWorld results.

Code is not yet released; Perplexity says open-source release is forthcoming. All baseline results are self-reported from the authors' own hyperparameter-tuned runs. The open question is whether the 6.5× token efficiency replicates beyond Qwen3 and DeepSeek-V2-Lite.

Sources

• Primary: A good teacher meets the student where they are
• Reported by: MarkTechPost: Perplexity and NVIDIA Researchers Introduce JOLT: a Joint Teacher-Student Reinforcement Learning Method