NVIDIA Nemotron 3.5 Super VL: 120B MoE Runs on a Single GPU

NVIDIA released Nemotron 3.5 Super VL on October 15, 2026: a 120B-parameter mixture-of-experts vision-language model under the OpenMDW-1.1 license with 12B active parameters per token. The NVFP4 checkpoint fits on a single DGX Spark, one H200, two H100s, or one GB200. For teams evaluating open multimodal agents, that combination of scale and deployment surface is the hardware-budget story, not an incremental bump.

The context window reaches 1 million tokens, placing Super VL in the long-context tier where capability has become a pricing and architecture problem. NVIDIA ships NVFP4 and BF16 checkpoints, plus open datasets, training recipes, and RL environments in NeMo Gym.

Architecture and inference stack

The backbone interleaves Mamba-2, mixture-of-experts, and attention layers; each MoE layer carries 512 routed experts. A C-RADIO vision encoder and connector translate images, document pages, and MP4 video clips up to two minutes into backbone representations, and remain in BF16 after quantization. The model was distilled from Nemotron 3 Ultra and, according to NVIDIA, trained on about 26 trillion tokens; post-training used roughly 3.8 million samples plus reinforcement learning across dozens of environments, and continued pretraining added 10 billion Indic tokens.

Four techniques drive throughput. Native multi-token prediction drafts several tokens and verifies them in one forward pass. LiLiCorr, the DGX Spark speculative drafter, delivers up to 2.3x faster token generation per NVIDIA. NVFP4 quantization-aware distillation applies W4A4 on Blackwell and W4A16 elsewhere. Context reuse keeps attention caches and Mamba states across agent turns. NVIDIA reports 600+ output tokens per second on Artificial Analysis agentic tasks via a Baseten endpoint and a 5x throughput gap over Inkling Small on that chart; both figures are vendor-reported.

Benchmark performance

All scores below are NVIDIA-reported from the NVFP4 checkpoint. Harnesses and benchmark versions differ across models; treat point gaps with caution.

On agent harness evaluations, NVIDIA reports Super VL beats Nemotron 3 Ultra — about 4.6x more total parameters — with 63.44% on Hermes Bench versus 56.33% and 87.05% versus 84.3% on LangChain evals. The 17.78 tau3-bench banking score is a clear floor and signals that long-horizon financial agent tasks remain a weak spot at this scale.

Deployment options

Supported runtimes are vLLM, SGLang, and TensorRT-LLM on Blackwell and Hopper; local paths include llama.cpp and Unsloth. The model is on Hugging Face, OpenRouter, ModelScope, and as an NVIDIA NIM. On DGX Spark, NVIDIA recommends LiLiCorr with seven speculative tokens; on H200 and two H100s, MTP with three draft tokens; on GB200, MTP with five speculative tokens and full W4A4 NVFP4. Reasoning toggles via enable_thinking. Production Dynamo recipes add KV-aware routing; on 16 B200 GPUs at 50 tokens per second per user, NVIDIA reports 8x more users served and 85% lower first-token latency.

AI Mastery analysis

The 12B active parameter count is the anchor, not 120B. MoE routing means per-token compute and bandwidth requirements sit closer to a 12B dense model, which is what makes single-GPU deployment viable despite the 120B label. But the full weights must still reside in memory, so the NVFP4 footprint—not the active parameter count—is the actual constraint. Non-Blackwell deployments should plan for W4A16 rather than W4A4, which changes throughput and memory pressure.

The tau3-bench banking score of 17.78 deserves attention. A model that scores 89.90 on GPQA-Diamond but 17.78 on long-horizon financial agent tasks suggests production failures will surface in architecture and task design, not headline model numbers. The 1M-token context window also adds surface for injected or malformed content to influence tool calls, an issue at the shared-surface layer of agent security rather than model logic.

The simultaneous Bolt launches sharpen the picture: Bolt 74B-A9B fits in 24GB via native 2-bit quantization for RTX and DGX Spark systems, and Bolt E4B (4GB) arrives October 23 for Jetson and embedded targets. That is a tiered open-weight stack from embedded to GB200. The open question is whether architectural specificity keeps outperforming raw GPU scaling at each tier once third-party evaluators get sustained access.

Most significantly, this is a hardware economics release: frontier-class multimodal agents, open weights, and a single-GPU path on shipping hardware change buy-versus-build for teams paying per-token API costs. Benchmark gaps against closed competitors are real; so is the opposite gap in operational control and data privacy.

Sources

• Primary: nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-NVFP4 · Hugging Face
• Reported by: MarkTechPost: NVIDIA Releases Nemotron 3.5 Super VL, a 120B Open Multimodal Agent Model