IBM Granite 4.2: 30B Model Hits 57.0 on SWE-Bench Verified

IBM released Granite 4.2 on August 25, 2026 — its first family of dense, decoder-only reasoning LLMs — in three sizes: 3B, 8B, and 30B. All three are pre-trained from scratch on approximately 15 trillion tokens, supervised fine-tuned on chain-of-thought and agentic-trajectory data, then post-trained through a multi-stage reinforcement learning pipeline. The release is Apache 2.0 licensed and targets developers who want open-weight reasoning models they can serve, fine-tune, or embed inside agentic harnesses without proprietary entanglement. For teams already thinking about how pipeline architecture drives inference-time gains, Granite 4.2's staged RL curriculum is a concrete example of that pattern applied at training time.

Architecture and Pre-Training

Every model uses grouped query attention with 40 attention heads and 8 KV heads, SwiGLU feed-forward layers, RMSNorm with ε = 1e-5, RoPE position embeddings with θ = 10,000,000, separate (non-tied) input/output embeddings, and bfloat16 precision. The 3B and 8B models differ primarily in MLP hidden size (8,192 versus 12,800) while both use 40 transformer layers; the 30B model scales to 64 layers and an MLP hidden size of 32,768. All three use a 131,072-token sequence length at the SFT stage. Pre-training uses a five-phase schedule: phases 1–2 cover broad web-scale pre-training, phases 3–4 apply progressively higher-quality data annealing, and phase 5 performs long-context extension to 512K tokens.

SFT Data Pipeline

The supervised fine-tuning corpus totals approximately 7.2 million samples, or roughly 100 billion tokens, of which about 65 billion are trainable. The mix splits 31.6% agentic and 68.4% non-agentic. Within the agentic portion, software engineering tasks dominate at 69%, followed by tool calling (12.1%), terminal use (8.0%), math (3.5%), search (0.8%), and action (0.2%). Trajectories were generated across scaffolds including OpenHands, OpenCode, Terminus-2, SWE-agent, OpenResearcher, MiniSWE, OpenSeeker, EnvScaler, Gemini CLI, Hermes, Codex, and Goose. Non-agentic categories include instruction following (18.8%), coding (18.8%), math (14.6%), multilingual (7.0%), science (5.4%), reasoning (3.0%), and safety (0.8%).

Quality control normalizes samples into a consistent OpenAI Chat format, then scores them using GPT-OSS-120B and Gemma 4 as LLM judges. Samples with hallucinated content, invalid tool interactions, or calls to undefined functions are removed. Deduplication uses SHA-256 hashes over the concatenation of the tools and messages fields, applied within and across data sources. The final corpus is globally shuffled before being partitioned into equally sized Parquet shards. SFT training runs for approximately two epochs at a constant learning rate of 1.0e-5 after a 2.5% warm-up, on 32–128 nodes of 4× Grace/GB200 hardware, using tensor-parallel degree 2 and context parallelism of 2 or 4 depending on model size. The 30B model receives a second SFT phase focused on agentic coding — upsampling SWE and coding data while retaining approximately 16% replay from the original corpus — trained for roughly one additional epoch at 3.0e-6.

Multi-Stage Reinforcement Learning

The post-training pipeline is a sequential curriculum of independent GRPO runs, each warm-starting from the previous stage's checkpoint. The algorithm is asynchronous: a generation pool samples responses into a shared buffer continuously while the trainer pulls batches and streams updated weights back without pausing generation workers. A limit prevents generators from drifting more than a single update behind the trainer, bounding off-policy exposure; truncated importance sampling then clamps the log-probability ratio to prevent stale tokens from dominating updates. Advantages are group-relative with a leave-one-out baseline, eliminating the need for a separate value network.

The 3B model follows a shortened ladder — RLVR (two rounds), a code booster, and RLHF. The 8B and 30B models add an agentic RL block (SWE → Terminal → Search) between the skill boosters and final RLHF, and the 30B runs RLVR for three rounds. All agentic stages use real sandboxed environments: SWE tasks run inside per-repo container images driven by OpenHands; the terminal stage executes multi-step shell tasks through the Harbor/Terminus-2 harness with rollouts spanning up to 64 environment turns; the search stage answers multi-hop questions using live web-search tool calls scored by an LLM judge. RLHF is the final stage for every model, using a generative reward model for preference plus a safety reward covering jailbreak resistance, with the highest KL penalty in the pipeline (0.05) and a reasoning-length penalty to suppress verbosity acquired in earlier stages.

Results

Quantization and Deployment

IBM released four quantized variant families alongside the base models. FP8 uses dynamic per-channel weights and per-token activations with no calibration. NVFP4 and MXFP4 are GPTQ-calibrated on 2,000 samples drawn from the SFT dataset, with a 2K maximum context length during calibration. GGUF conversion uses the canonical llama.cpp toolchain and ships 14 quantization levels from Q2_K through Q8_0. All sizes are served via an OpenAI-compatible endpoint through vLLM or SGLang, emitting tool calls in OpenAI function-calling format and integrating directly with OpenCode, Pi, and OpenHands without custom adapters.

Training ran on an NVIDIA GB200 NVL72 cluster hosted by CoreWeave, using a 72-GPU NVLink domain and a non-blocking Fat-Tree NDR 400 Gb/s InfiniBand fabric for inter-rack connectivity. The software stack runs in containerized .sqsh images built on an NGC PyTorch base (Ubuntu 22.04, CUDA 12.8, Python 3.12), with RL stages isolated in a separate NeMo-RL container.

Granite 4.2 operationalizes a reproducible, modular RL curriculum in which each stage is an independent GRPO run with a defined reward type, and agentic capability is a plug-in block rather than a property of the base model. For teams evaluating small open-weight models for production agentic workflows, the 8B's SWE-Bench Verified score of 47.67 and its full agentic-RL treatment make it a credible on-premises alternative to hosted offerings.