Kyutai CADRE: World Models at 20 fps With Under 230 ms Lag

Kyutai has published CADRE (Causal Diffusion Representation autoencoders), an open family of video autoencoders built for action-conditioned world models. The release covers seven autoencoder variants, four extended multi-resolution versions, and four trained world models. Code is MIT-licensed on GitHub; weights are on Hugging Face under CC BY-NC 4.0, with the frozen DINOv3 encoder under Meta's DINOv3 License. Kyutai reports that world models trained on CADRE latents sustain 20 fps on a single H100 and react in under 230 ms—a threshold the paper identifies as the noticeability limit for self-generated movement lag.

Most video autoencoders are built for text-to-video generation, where temporal compression is cheap because no real-time interaction loop exists. Pack τ frames into one latent and every downstream action waits for the chunk to close. At τ = 8 and 20 fps, the chunk alone adds 400 ms before generation or decoding starts—already beyond the 230 ms limit. As our coverage of infrastructure rewrites not model weights driving 2026 AI gains notes, this kind of structural bottleneck separates deployable systems from demos.

CADRE uses the MIRA recipe. A frozen DINOv3 ViT-B/16 with 86M parameters extracts per-patch features, and one trained 3D convolution compresses τ frames and s×s pixels into a 64-channel latent. That bottleneck is the only component that changes across the seven members, which include cells from 1×48×48 to 8×16×16. The decoder is a causal diffusion transformer with 24 blocks and about 476M parameters per autoencoder, trained by progressive self-distillation to decode in one or two steps in a single stage without an adversarial loss. Because decoding is causal, it can begin when the first latent is ready instead of waiting for a full sequence. Training used roughly 76,000 hours of video from 23 public sources spanning web, robotics, driving, and game footage. The world model is a 1.2B-parameter causal diffusion transformer, trained separately on each of four environments.

MarkTechPost's comparison of Kyutai's paper tables lists the following single-H100 figures at BF16, batch size one, one decoding step:

Latency, fps, FVD, ARR and PSNR are self-reported by Kyutai from Tables 4, 6 and 7; MarkTechPost compiled the comparison. CADRE results use one decoding step.

Kyutai also publishes measured demo throughput. On one H100 serving two active users, it reports about 14 fps per user with local input p95 latency of 207–212 ms; one active user measured 20 fps and 162 ms. Network transit and browser rendering are excluded, and two users do not meet the 19 fps / 200 ms target.

Kyutai's ablation trained the same 1.2B world model on each autoencoder across robot manipulation (ABC-130k), human manipulation (EgoDex), driving (PhysicalAI-AV), and indoor navigation (RealEstate10K), generating 4.8 seconds after 2 seconds of real-video context. MarkTechPost summarizes four findings. First, one-step diffusion decoding at 2×32×32 reduces FVD by 1.4–2.5× relative to MIRA's perceptual regression decoder while preserving action following. Second, patching time inside the world model loses detail: at 8 frames per latent, robot manipulation ARR drops from 0.72 to 0.56. Third, camera motion decides the cheap compression axis—static cameras tolerate up to 8 frames per latent, while navigation FVD rises by up to 64% for longer latents. Fourth, coarse spatial tokens hurt motion before image quality: at 2 frames per latent, 32× spatial compression raises robot FVD from 39.4 to 95.2 while FID moves only from 8.53 to 8.92.

AI Mastery analysis

The strongest contribution is controlled experimental design, not a single best model. Holding the 1.2B world model fixed and changing only the bottleneck convolution isolates compression geometry as a causal variable. The result that reconstruction PSNR misleads is important: Wan 2.2 leads at 35.02 dB PSNR but ranks last on indoor rollout FVD, so selecting autoencoders by reconstruction metrics alone would be wrong.

Latency arithmetic exposes a deployment constraint. The 230 ms target leaves little room, and network transit plus browser rendering are explicitly outside Kyutai's measurements; the shared demo already queues users beyond two active connections. Teams will need to co-locate inference with input capture or accept perceptible lag, an infrastructure-layer choice covered in our AI deployment layer analysis. The CC BY-NC 4.0 weights also rule out commercial deployment of the checkpoints as-is: production users would have to train their own weights from MIT-licensed code. The multi-resolution variants train to 400k steps, the last 40k on multiple resolutions, and reconstruct better than the paper models, but their latent spaces differ, so they cannot be used directly with the released world models.

CADRE supplies a reproducible latency-quality frontier with real hardware numbers and a systematic compression ablation. Whether these findings transfer to generalist multi-environment models remains open.

Sources

• Primary: GitHub - kyutai-labs/cadre-autoencoders-and-world-models: CADRE: causal video autoencoders and real-time interactive world models. Code for
• Reported by: MarkTechPost: Kyutai CADRE: Video Autoencoders That Cut World Model Lag Below 230 ms