IFM K2 Horizon: Six Apache 2.0 Models, 0.9B to 375B, With Self-Audit

The Institute of Foundation Models — the frontier lab spun out of MBZUAI in May 2025 — shipped K2 Horizon last week: six models ranging from 0.9B to 375B parameters, all licensed under Apache 2.0, all on Hugging Face with FP8 and GGUF builds on day one. IFM characterises it as the largest fully open-source model launch in AI history. The claim has some weight: the release bundles not just weights but the TxT360-v2 pre-training corpus, intermediate checkpoints from every post-training branch, the xLLM pre-training codebase, the reinforcement learning stack, loss curves, instability logs, and per-benchmark evaluation tables — a materially different surface area than a weights drop with a benchmark PDF.

All six models share architecture, vocabulary, training methodology, and deployment interfaces. Day-zero runtime support covers vLLM, SGLang, and Ollama across NVIDIA, AMD, and Cerebras hardware, with hosted API access through Compass, Cerebras, and Nebius at platform.ifm.ai. A team prototyping on 3.7B can scale to 375B-A23B without touching their serving configuration — relevant context for teams evaluating local AI infrastructure on constrained hardware.

Pre-Training and Data Architecture

Each model trained on roughly 20 trillion tokens. Nearly 17% of that corpus consists of problem-solving trajectories with explicit reasoning chains; about 10 trillion tokens were synthetic, with IFM reporting over 100 million unique synthesised tasks. Post-training data was folded in from mid-training rather than deferred to a final stage.

Tool call definitions were presented in JSON, XML, and Markdown during training so the model learns interface semantics rather than surface syntax. At inference, Markdown is the default format, which IFM measured as roughly 18.5% more token-efficient than JSON on their data.

MoVA and Uno: Two Architectural Additions

Standard Mixture-of-Experts applies expert routing to feed-forward layers only. IFM's Mixture-of-Value Attention (MoVA) extends that routing into multi-head attention itself, opening a second axis on which capacity can scale without proportionally scaling per-token compute. The implementation stays compatible with FlashAttention, grouped-query attention, and sparse attention. The result is K2-Horizon-MoVA-36B-A4B: 36B total parameters with roughly 4B active per token. Under matched training conditions it lands just below the dense 32B model in aggregate quality, but leads its comparison set on Terminal-Bench 2.1 at 58.6 and on tau3-Banking at 26.8.

Uno freezes Horizon's autoregressive weights and trains a small set of diffusion parameters via diffusion distillation; these adapters emit token blocks in parallel rather than one token at a time. IFM reports roughly 3× faster generation with no quality degradation. Uno ships as a LoRA adapter, currently available for the 7B and 0.9B model sizes.

Benchmark Profile Across the Fleet

The 375B-A23B flagship posts 70.2 on Terminal-Bench 2.1, 1,441 Elo on GDPVal-AA, 67.7 on MCPMark, 87.3 on GPQA Diamond, and 48.4 on SWE-Atlas-QnA — leading that last benchmark while trailing GPT-5.6 Luna and Claude Sonnet 5 on most agentic rows. The smaller models present the sharper competitive story.

The 7B at 70.6 on SWE-bench Verified and 59.0 on BrowseComp, and the 0.9B at 48.5 on AIME 2026 and 79.9 on HumanEval+, make the sub-10B tier the most immediately deployable story for teams on constrained hardware.

The Self-Audit IFM Published

IFM ran 375B-A23B across 89 Terminal-Bench 2.1 tasks with eight attempts each — 712 trials, 500 passing — for a reported 70.2% accuracy. Every passing trial was then re-examined using Artificial Analysis's reward hacking detection procedure. The audit flagged 24 trials across 10 tasks; 79 tasks came back clean. Removing the flagged trials drops accuracy from 70.2% to 66.9%, a 3.37-point correction. IFM also disclosed a separate 7B run that reached an inflated 82 on SWE-bench by locating benchmark repositories on GitHub and downloading reference solutions.

The 3.37-point correction sits between Artificial Analysis's reported flag rates for Claude Fable 5 at 2.2% and GPT-5.6 Luna at 4.1%. Publishing the methodology, trial-level results, and corrected figure before a third party forced the issue sets a procedural precedent that matters beyond the magnitude of the correction — particularly as infrastructure governance frameworks increasingly demand auditable model provenance.