NVIDIA Nemotron 3.5 Lightning: 30B Parameters, 3B Active

NVIDIA released Nemotron 3.5 Lightning on August 11, 2026, targeting a specific inefficiency in production agent systems: the cost of routing every step through a frontier reasoning model, even when that step is a tool call, file read, or output validation. The model carries 30 billion total parameters under the OpenMDW 1.1 license but activates only 3 billion per token via a Mixture-of-Experts routing layer — a configuration that lets it operate closer to a much smaller model on inference cost while retaining the representational depth of a larger one. For engineers already thinking through what the agentic era demands of production infrastructure, this release is a concrete implementation of an architectural thesis that has been debated mostly in the abstract.

The implied deployment pattern is tiered: an expensive frontier or reasoning model handles planning, then Nemotron 3.5 Lightning handles the execution loop. NVIDIA claims up to 4x output speed over comparable-sized models, attributing that figure to the combination of MoE sparsity, Mamba-2 sequence processing, and speculative decoding — not any single optimisation.

Architecture

The hybrid design stacks Mamba-2 state-space layers, sparse MoE, selective attention, and Multi-Token Prediction into a 52-hidden-layer network. The MoE layer exposes 128 routed experts plus one shared expert, with six routed experts selected per token — so at each forward pass, the active parameter count sits near 3 billion rather than the full 30 billion. This is the primary throughput lever.

Mamba-2 handles most sequence processing without paying full quadratic attention costs across every layer. NVIDIA preserves selected attention layers for positions where global token interaction is genuinely necessary — long retrieved documents, multi-step tool trajectories, agent memory — rather than applying attention uniformly. Multi-Token Prediction layers, which received a dedicated continued-pretraining stage, train the model to predict several future tokens simultaneously, enabling speculative drafting at inference.

NVIDIA ships three speculative decoding strategies: MTP (integrated directly into the model weights, recommended for medium-to-high concurrency), DSpark (a dedicated draft model tuned for DGX Spark and lower-concurrency data-centre inference), and DFlash (an additional draft model intended for benchmarking against the other two). Single-GPU deployment is validated on a DGX Spark GB10 or H100, with quantisation options extending support across Blackwell, Hopper, and Ampere hardware. Supported quantisation formats are NVFP4 and W4A16; the full-precision checkpoint ships in BF16.

Benchmark Results

NVIDIA evaluated both BF16 and NVFP4 checkpoints across knowledge, reasoning, coding, agent, instruction-following, and long-context tasks using a NeMo Gym and NeMo Evaluator harness, with published recipes for reproducibility. The headline finding is that NVFP4 scores remain within a few points of BF16 across nearly every task — the configuration most teams would deploy for throughput.

SWE-bench Verified at 51.56 in BF16 is the coding-agent figure most practitioners will scrutinise. The NVFP4 result of 52.80 on that task reflects benchmark noise rather than a genuine quantisation gain, but confirms NVFP4 does not degrade agent task performance meaningfully. Recommended inference parameters are temperature 1.0 and top-p 0.95; reasoning is supported and configurable via a reasoning_budget field and an enable_thinking flag.

Pricing and Access

The model is open-weight, so pricing depends on which access route a team selects. As of August 12, 2026, NVIDIA's endpoint at build.nvidia.com is free for prototyping with a 1M-token context. OpenRouter lists a free route at 1M context and a standard paid route at $0.05 per million input tokens and $0.20 per million output tokens, though the standard route caps context at 262K. Fireworks serverless pricing matches OpenRouter's rates — $0.05 input, $0.01 cached input, $0.20 output per million tokens — also at a 262K context window. Self-hosted deployments via Ollama or vLLM carry no per-token model fee, with the full 1M-token context available to teams willing to own the infrastructure cost.

The API follows the OpenAI-compatible interface pattern, with model identifier nvidia/nemotron-3.5-lightning-30b-a3b. Primary language support targets English and programming languages; Spanish, French, German, Italian, and Japanese are additionally supported.

Infrastructure vendors are beginning to productize the software-extraction approach to AI efficiency at the model level itself — not just through quantisation tooling or serving frameworks, but through architectures explicitly designed for a role inside a multi-model stack. Nemotron 3.5 Lightning does not compete with frontier reasoning models for hardest-problem tasks; it competes with the cost and latency overhead of using those models for work that does not require them.