Jalapeño Beats GB200/GB300 by 1.9× Efficiency, 3.6× Latency

OpenAI published benchmark results for Jalapeño on August 25, 2026, marking the first concrete performance disclosure for its custom inference silicon. Jalapeño was tested on the public InferenceX benchmark from SemiAnalysis against commercially available accelerators across three open-weight models — GPT‑OSS 120B, DeepSeek R1 670B, and Kimi K2.5 1T — and placed on the Pareto frontier for throughput-per-kilowatt versus latency in every case.

Architecture

Jalapeño targets the phase asymmetry of transformer inference: prefill is compute-bound; decode is memory-bandwidth-bound. Systems optimised for one phase stall during the other, and inter-chip communication adds latency when KV cache state must migrate between resources. OpenAI's approach makes the chip a fungible accelerator that handles both phases without penalty. Model state, including the KV cache, is explicitly placed to remain local while the chip activates the appropriate mix of compute, memory, and networking for whichever phase is active. The network fabric is integral rather than peripheral: its domain is large enough that an entire request can complete within one connected system. This design maps directly onto pipeline architecture arguments gaining traction in 2026, where the efficiency of the serving stack matters as much as the model itself.

Development moved from initial design to tapeout in nine months. OpenAI's own models shortened verification loops, optimised arithmetic circuits, and iterated on workloads. Using Codex with GPT‑Astra, the team brought three open-weight models not in the original production plan to high performance within two months. For selected GPT‑OSS attention and mixture-of-experts blocks, AI-generated kernel implementations ran 1.5 to 1.8 times faster than existing human-expert-written versions — scoped to those specific blocks, not the full model.

InferenceX Results

Jalapeño carries a rated package TDP of 700 W; measured sustained power during testing stayed at or below 550 W. Comparison systems were NVIDIA GB200 (rated 1,200 W) for GPT‑OSS 120B and GB300 (rated 1,400 W) for DeepSeek R1 and Kimi K2.5. All throughput-per-kilowatt figures use each vendor's published chip power rating for normalisation.

The most striking single figure is the matched-TBT throughput comparison for DeepSeek R1: at the comparison system's best time-between-tokens rate of 169.41 tok/s/user, Jalapeño delivered 12,258 mixed tokens/kW against 118 — a 104.3× difference in throughput-per-watt at that operating point. For GPT‑OSS 120B at 535.28 tok/s/user, the equivalent figure is 22,935 versus 427 mixed/kW, a 53.7× gap. These numbers reflect what happens when the latency constraint is held fixed: the efficiency gap widens dramatically because the comparison hardware had already reached its effective operating limit.

In internal testing on frontier OpenAI models — not the public benchmarks reported above — Jalapeño's advantage widened further, consistent with an architecture that gains leverage as model size and demand intensity increase.

Deployment and Roadmap

Jalapeño will begin deployment within OpenAI's compute infrastructure before the end of 2026. Gen 2 is described as deep in development; Gen 3 is taking shape. OpenAI explicitly states it will continue deploying NVIDIA accelerators and hardware from other partners for both training and inference workloads alongside Jalapeño.

The chip enables three operating-point shifts: ultra-fast-mode inference at efficiencies previously achievable only in fast mode; fast-mode inference at efficiencies previously requiring batched operation; and improved efficiency for batched workloads generally.

Custom silicon increasingly determines which providers can absorb inference cost at scale without proportional revenue growth — a dynamic covered in the broader hardware strategy landscape. A 700 W chip outperforming a 1,200–1,400 W competitor by 1.5–1.9× on useful work per watt, while delivering 1.7–3.6× lower latency across models from 120B to 1T parameters, is a substantive result. The multigenerational roadmap signals an infrastructure bet OpenAI intends to compound across generations, not a single-cycle demonstration.