DeepSeek-V4.1-Flash: 890 Bytes Per Token, 437x Smaller KV Cache

DeepSeek AI has released DeepSeek-V4.1-Flash, a 552B-parameter multimodal Mixture-of-Experts model that compresses its global KV cache to 890 bytes per token — roughly one-quarter of DeepSeek-V4-Flash and 437 times smaller than DeepSeek-V1. The release targets a problem structurally painful for long-horizon agent workloads: repeated prefills over million-token contexts saturate HBM bandwidth and SSD capacity faster than hardware procurement can compensate. Rather than scaling around the bottleneck, DeepSeek engineered through it, stacking three interlocking mechanisms — a split encoder-decoder architecture, a layered sparse attention scheme, and FP4 KV quantization — to shrink the cache without abandoning expressivity. Weights ship under MIT license with vLLM, SGLang, and Transformers compatibility on Hugging Face, and a public API exposes low, high, and max reasoning tiers mapped to fixed scalar effort values of 50, 75, and 100 respectively.

Causal Encoder-Decoder and Prefill Halving

The 40-layer backbone divides into a 20-layer causal encoder and a 20-layer decoder, a structure inspired by YOCO. During prefill, decoder global KV is not computed layer by layer — it is projected from the final encoder hidden state using per-layer projection weights. Prompt tokens never pass through the decoder, which cuts prefill compute to roughly half a full forward pass and activates only 8B parameters per token. Decode activates 16B parameters per token across all 40 layers, drawing on 1 shared plus 6 routed experts. Single-token decode FLOPs rise by only 1.25x when context grows from 4K to 1M tokens. Decoder SWA states are rebuilt via Decoder SWA Bounded Replay — replaying only the last 128 prompt tokens rather than re-running full layers. This architectural specificity over hardware scaling is what separates efficiency gains achieved through engineering from those that rely on adding silicon.

CSA2: Sharing Main KV, Indexer K, and Top-K Indices Across Layers

DeepSeek-V4.1-Flash replaces V4's mixed CSA and Heavily Compressed Attention with pure Compressed Sparse Attention 2 (CSA2), which attacks cache size along the layer axis by statically assigning each layer one of three modes:

• Full: computes its own main KV, projects indexer K from it, and selects fresh Top-512 indices. Also constructs a hierarchical candidate pool of up to 16,384 positions (2,048 blocks of 8) in decoder layers.
• Reindex: reuses main KV and indexer K from the preceding Full layer but applies its own indexer Q to rescore the candidate pool, producing fresh Top-512 selections without recomputing main KV.
• Reuse: skips the indexer entirely and attends using Top-K indices from the nearest Full or Reindex layer. Runs with 15 kernels in prefill and 11 in decode.

Every layer, regardless of mode, computes its own main Q and SWA KV. The 18 CSA2 encoder layers use a 2-in-3 compression ratio across three groups of 6, each group containing 1 Full layer and 5 Reuse layers. The 20 decoder layers use a 1-in-4 compression ratio across five groups of 4: the first group is 1 Full plus 3 Reuse, and the remaining four are 1 Reindex plus 3 Reuse.

FP4 KV Quantization and Cache Lifetime Management

The main KV cache uses E2M1 format with one E4M3 scale per 16 channels, following the NVFP4 scheme without its global scale. This quantization is introduced through quantization-aware training during post-training and approximately halves storage relative to V4-Flash's FP8 cache. SWA KV is no longer persisted to SSD; it lives in a distributed pool carved from 10% of host DRAM with a TTL measured in minutes. Global KV holds a guaranteed 72-hour lifetime. Combined, these choices mean the real serving bottleneck shifts to cache infrastructure policy — a distinction that matters when provisioning multi-tenant long-context deployments.

Additional changes include Single-Pass mHC, which shifts input-mixing coefficients by one block to allow a fused Mega-mHC kernel to halve activation memory traffic; the Engram conditional memory module active at layers 1 and 14 across 196B additional parameters; and DSpark speculative decoding, trained post-pre-training with the backbone frozen.

Benchmark Performance

Pre-training covers 45T multimodal tokens at a 7:1 text-to-multimodal ratio, with sparse attention trained from scratch at 64K sequence length and extended to 1M at 34T tokens. Post-training relies on large-scale synthesis of verifiable agent tasks, RL across six heterogeneous scaffolds (Claude Code, Codex, OpenCode, Pi, mini-SWE, DeepSeek Harness), and on-policy distillation from over 40 teachers.

At effort 25, the model averages 67.1 across eight reasoning benchmarks and scores 66.0 on DeepSWE v1.1; at effort 100, those figures rise to 76.3 and 74.2 respectively, with output token cost approximately 2.5x higher. The base model matches DeepSeek-V4-Pro-Base on world knowledge and coding while consuming one-third of V4-Pro-Base's total parameters and one-quarter of its activated parameters.

The 437x KV cache reduction across four generations — achieved without dense-attention warm-up training and under an open MIT license — sets a concrete reference point for what systems-level engineering can deliver against parameter scaling. Teams evaluating the model for agentic deployments should treat the 890-byte-per-token figure as a hard constraint that reshapes SSD provisioning, DRAM allocation, and KV lifetime policy in production.