Inkling 2 Mini: 26B Looped MoE Runs on a 32GB Mac, 79.2% SWE-bench
In this article
Thinking Machines Lab has released Inkling 2 Mini, an open-weight looped transformer with 26.4B total parameters. The model reuses its middle layers three times per token instead of adding weights, leaving a checkpoint near 20GB that fits a 32GB Mac, one NVIDIA DGX Spark, or any GPU with 32GB or more VRAM. For local agent builders, that memory envelope is the story. It shifts the local AI workload decision from capability to architecture, with Thinking Machines reporting 79.2% on SWE-bench Verified from a checkpoint in that memory class.
The looped MoE architecture
Inkling 2 Mini stores 24 layers in four blocks of six. The first and last blocks execute once; the two middle blocks execute three times with shared weights, producing 48 layer applications per token from 24 layers of stored weights. Each pass after the first receives a message carried by what Thinking Machines calls Attention Residuals plus a learned gate, so the loops are not identical recomputations.
Feed-forward layers hold 128 experts with 8 routed per token. Training started at 384 experts per layer, then pruned two-thirds at midtraining. The team kept the 128 most-used experts and healed the pruned model with a 1T-token distillation run from a larger internal Inkling model. Expert layers are trained and deployed in NVFP4, and the last four layers use MXFP8, an arrangement Thinking Machines says removes quantization conversion loss.
Looping delivers a 1.51x to 1.91x compute-equivalent gain over an unlooped model, according to the lab. The tradeoff is specific: the unlooped baseline needed 7.5% less training compute, but 140% more inference compute to match the looped model's loss on 512k-token agentic traces. Looping the last layers instead of the middle produced only 1.24x to 1.36x, and the middle-loop variant won 27 of 28 evaluations.
Comparative benchmarks
Inkling 2 Mini's benchmark figures are self-reported by Thinking Machines; competitor values in the table come from each vendor's own model card, and harnesses differ, so cross-vendor gaps are directional. Coding is strongest: 79.2% on SWE-bench Verified (bash only), 77.9% on SWE-bench Pro v2 (Hard), and 48.2% on DeepSWE v1.1, ahead of Qwen3.8-27B's 42.2% from Qwen's card. Agentic search posts 79.0% on BrowseComp with compaction and 82.7% on MCP Atlas. Documents and vision score 92.8% on OmniDocBench v1.6 and 74.9% on MMMU Pro; reasoning is 86.6% on GPQA Diamond. Weak spots are knowledge recall and constrained multi-turn work: 15.2% on HLE-Diamond, 50.0% on MultiChallenge, 33.4% on ComplexConstraints v1.1, and -8.6 on AA-Omniscience with a 63.4% non-hallucination rate. On the Artificial Analysis Index, Thinking Machines reports 20.1, tied with Qwen3.8-27B.
| Inkling 2 Mini | Qwen3.8-27B | Qwen3.6-35B-A3B | Gemma 4 26B A4B | |
|---|---|---|---|---|
| Architecture | Looped MoE | Dense | MoE | MoE |
| Total / active params | 26.4B / 3.4B (5.4B effective) | 27B / 27B (dense) | 35B / 3B | 25.2B / 3.8B |
| Context | Up to 1M tokens | 262,144 native, up to 1M | 262,144 native, up to 1,010,000 | 256K |
| SWE-bench Verified | 79.2% (bash only) | Not reported | 73.4% | Not reported |
| DeepSWE v1.1 | 48.2% | 42.2% | Not reported | Not reported |
| Terminal Bench v2.1 | 71.6% | 73.0% | Not reported | Not reported |
| GPQA Diamond | 86.6% | 89.2% | 86.0% | 82.3% |
Efficiency and deployment
Thinking Machines reports a Stanford Intelligence per Watt score of 0.703 on a DGX Spark, against 0.430 for Qwen3.6-35B-A3B and 0.159 for Gemma-4-26B-A4B. The Safetensors checkpoint is roughly 20GB. Listed setups are one DGX Spark, Apple silicon with 32GB or more memory, and GPUs with 32GB or more VRAM. GGUF builds ship via Unsloth and llama.cpp; serving paths include Transformers, vLLM, SGLang, Ollama, NVIDIA Dynamo, and TokenSpeed. Tinker pricing for Mini is not disclosed in the launch materials, while Hugging Face's provider data lists OpenRouter hosting via PrimeIntellect at $0.075 input and $0.30 output.
AI Mastery analysis
The architectural claim is specific and testable: middle-layer looping returns a 1.51x to 1.91x compute-equivalent gain, versus 1.24x to 1.36x for terminal-layer looping. The gated message between passes is a concrete mechanism rather than parameter-count marketing. This fits the architectural specificity pattern we track: a 20GB artifact under Apache 2.0 changes the portability problem. The bottleneck moves from model access to planning, context management, and tool-calling infrastructure.
What the launch does not resolve is local runtime. The source material reports no tokens-per-second on the 32GB Mac target, only a DGX Spark efficiency figure and OpenRouter throughput around 30 to 41 tokens per second from Hugging Face provider data. Treat the self-reported 79.2% SWE-bench score as a capability bound, not a local latency guarantee. The weak knowledge-recall scores also suggest pairing the model with retrieval rather than relying on it as a large knowledge store. Open-weight releases are now optimizing for the inference budget of a single workstation, with comparisons against dense 27B and MoE 35B peers rather than frontier APIs.
Sources
Frequently asked questions
What hardware do I need to run Inkling 2 Mini locally?
Thinking Machines lists three supported setups: one NVIDIA DGX Spark, Apple silicon with 32GB or more memory, and other GPUs with 32GB or more VRAM. The Safetensors checkpoint is roughly 20GB, and GGUF builds are available through Unsloth and llama.cpp for quantized deployments.
How does Inkling 2 Mini's looped architecture work?
It stores 24 layers in four blocks of six. The first and last blocks run once, while the two middle blocks run three times with shared weights, producing 48 layer applications per token. Repeated passes receive a learned gate message via Attention Residuals.
What are Inkling 2 Mini's coding benchmark scores?
In Thinking Machines' self-reported results, Inkling 2 Mini scores 79.2% on SWE-bench Verified using bash only, 77.9% on SWE-bench Pro v2 (Hard), and 48.2% on DeepSWE v1.1. The DeepSWE result is ahead of Qwen3.8-27B, which Qwen's own card lists at 42.2%.
Where does Inkling 2 Mini fall short?
Knowledge recall and constrained multi-turn work are the weak spots: 15.2% on HLE-Diamond, 50.0% on MultiChallenge, and 33.4% on ComplexConstraints v1.1. It also carries a -8.6 AA-Omniscience score with a 63.4% non-hallucination rate.
Does Inkling 2 Mini support tool calling and hosted inference?
Yes. Hugging Face's provider data lists OpenRouter hosting through PrimeIntellect with structured output and tool calling, at $0.075 input and $0.30 output. Serving paths include Transformers, vLLM, SGLang, Ollama, NVIDIA Dynamo, and TokenSpeed.
Related Reading
StepFun Opens 600B MoE Weights: 27B Active, 1M Context
StepFun's Step 5 Preview is now open weights: a 600B sparse MoE with 27B active parameters, 1M context, and a $1.00/$2.70 API.
JetBrains Mellum2.1: 2.5B Active Params Reach 47 SWE-bench Verified
JetBrains' 12B MoE model activates 2.5B parameters per token and lifts SWE-bench Verified from 2.0 to 47.0 after reinforcement learning in real repositories.
Cohere's 218B MoE Scores 83.6 on WMT26, Beats DeepL and Google Translate
North Small Translate activates 25B of 218B parameters per token, scores 83.6 on WMT26 across 50 languages, and runs on a single B200 in 4-bit mode.