Speculative Decoding

5 pieces on Speculative Decoding, including 1 step-by-step guide.

Guides

guides
architect • 2026-10-26
vLLMspeculative decodingEAGLEn-gramMTPGPU inference

Benchmark vLLM Speculative Decoding with EAGLE, n-gram, and MTP

Run vLLM's speculative decoding benchmark, configure EAGLE, EAGLE3, n-gram, custom draft models, and MTP, and interpret acceptance metrics.

Read more →

News & Analysis

news
Shan • 2026-09-20
OCRMixture of ExpertsSpeculative DecodingDocument ParsingOpen Weights

Jina-ocr-v1: 3.4B MoE Parses 2.57 Pages/sec on One A100

Jina AI's 3.4B MoE document parser posts 2.57 pages/sec on a single A100 — highest of 14 systems benchmarked — via lossless speculative decoding.

Read more →
news
Shan • 2026-08-24
Speculative DecodingCPU InferencevLLMIntel XeonLarge Language Models

DFlash Delivers 3.92x CPU Token Throughput on Qwen3.5-9B

DFlash in vLLM v0.25.0 hits 3.92x average token throughput on a Xeon 6975P-C CPU — a 74.40% cost cut with one config flag.

Read more →
news
Shan • 2026-08-22
Local AIOpen WeightsSpeculative DecodingAgentic Codingllama.cpp

Muse Glimmer 30B Hits 127 tok/s Locally via DFlash Speculative Decoding

Meta's Muse Glimmer 30B runs at up to 127 tokens/second on an RTX 3090 using llama.cpp, DFlash speculative decoding, and the Pi coding agent.

Read more →
news
Shan • 2026-08-20
Liquid AISpeculative DecodingInference OptimizationSmall Language ModelsOpen Weights

LFM2.5-DSpark Hits 3.18x GPU Speedup With Zero Output Change

Liquid AI's DSpark draft checkpoints deliver up to 3.18x throughput on H100 and 2.87x on M4 Max MacBook Pro, with bit-identical greedy output.

Read more →