NVIDIA Nemotron 3.5 Lightning: 30B MoE with 3B Active Parameters

NVIDIA shipped two artifacts on August 11, 2026: Nemotron 3.5 Lightning, a 30B mixture-of-experts model with 3B active parameters, and NeMo Switchyard, an open-source routing library that assigns each agent step to the most capable and efficient model available. The combination targets a structural inefficiency in long-running agents — tool calls, result validation, and subagent delegation consume the bulk of the token budget, yet most pipelines route all of that execution-layer traffic to frontier reasoning models priced and sized for orchestration-layer problems.

Both components ship under the permissive OpenMDW-1.1 license with open weights, training data, and recipes; commercial use is permitted. The hardware floor is a single DGX Spark (GB10) or a single H100, putting solo developers and seed-stage teams on the same footing as enterprise deployments. This is directly relevant to the emerging economics of agentic pipelines.

Architecture and Speed

Lightning's hybrid Mamba-2 + MoE + Attention architecture activates only 3B of its 30B parameters per token through a learned routing gate, giving it the representational capacity of a much larger dense model at a fraction of the per-token compute cost. Pre-training covered more than 20 trillion tokens using an NVFP4 recipe. Context length is 1M tokens. Lightning is the smallest member of the Nemotron 3 family; Nemotron 3 Ultra handles orchestration and planning while Lightning handles routine execution calls beneath it.

Speed comes from two compounding mechanisms. First, multi-token prediction was integrated during a dedicated pre-training stage and then reinforced with an MTP-boosting phase. Two external draft models ship alongside: DSpark, a semi-autoregressive drafter recommended for DGX Spark and low-concurrency data center workloads, and DFlash, which uses a lightweight block-diffusion model. Second, an NVFP4 checkpoint ships alongside BF16, with the same checkpoint serving Blackwell and Hopper natively and extending to Ampere through W4A16 kernels. NVIDIA reports up to 4x output speed versus similar-sized models.

Benchmarks

Published model card results across BF16 and NVFP4 checkpoints:

Recommended sampling parameters are temperature 1.0 and top_p 0.95. NVFP4 quantization causes negligible degradation on MMLU Pro (0.32 points), and GPQA Diamond actually improves by 0.13 points, making NVFP4 a practical default for Blackwell deployments.

NeMo Switchyard

NeMo Switchyard provides four router types, three of which require no manual tuning. An LLM classifier with session affinity maintains routing consistency across a conversation. A stage router reads recent tool activity to infer task complexity. An escalation router starts every session on the cheaper model and promotes to a frontier model only when an LLM judge detects sustained difficulty. The fourth, a prefill router, is tunable — it learns from the model's residual stream to predict which candidate will succeed before spending tokens on a full forward pass. The reference server accepts OpenAI, Anthropic, and Responses API requests, integrating with existing agent frameworks without protocol changes.

LangChain benchmarked Switchyard across 145 multi-turn agentic tasks from its internal deep agents suite over five runs. Routing between Nemotron 3.5 Lightning and Claude Opus 4.8 with the escalation router cut cost 74% versus a frontier-only baseline by sending just 7% of calls to the frontier model, at a measured roughly 6-point accuracy tradeoff. Separately, Cognition's implementation in Devin Desktop used staged routing between Opus 5 and Kimi K2.7, reaching 50.6% on FrontierCode Main at a $3.11 mean cost — within 2.8 points of Opus 5 accuracy at approximately 28% lower mean cost.

That tradeoff profile is comparable in spirit to what Sakana's Fugu orchestration model demonstrated with swappable frontier model ensembles, but Switchyard operates at per-step rather than per-task granularity, compressing savings further for high-frequency agent loops.

Availability

Weights are on Hugging Face and ModelScope; local inference runs on Ollama, LM Studio, llama.cpp, and Unsloth. Mid-market teams can use hosted inference through Baseten, Together AI, and Nebius; regulated enterprises can deploy fully on-premises. Named early adopters include CrowdStrike, Harvey, CodeRabbit, Fastino Labs, and Lila Sciences, spanning cybersecurity, legal, coding, finance, and life sciences workloads. The combination of a permissive license, a single-GPU deployment floor, and a quantized checkpoint that holds benchmark parity with BF16 removes most of the friction that slows open-weight adoption in production pipelines.