StepFun Opens 600B MoE Weights: 27B Active, 1M Context

StepFun made its Step 5 Preview model available as open weights on October 14, 2026, roughly three weeks after the API went live on September 20. Both FP8 and BF16 checkpoints are now on Hugging Face and ModelScope, giving engineers a self-hostable frontier-class model for agentic workloads. The architecture combines 600B total parameters with 27B active per token through a sparse Mixture-of-Experts design and supports context windows up to 1 million tokens.

Architecture and inference design

StepFun reports 92 transformer blocks arranged in a narrow, deep pattern. Attention alternates between gated sliding-window attention and full-attention layers in a 3:1 ratio: 69 sliding-window blocks interleaved with 23 full-attention blocks. The full-attention layers use Sparse GQA, with each KV group selecting up to 4,096 history tokens. Adjacent positions compress into blocks of 8, which StepFun reports cuts indexing cost by 8× relative to dense scoring. Attention coverage under this scheme is 78.06%, versus 79.09% for a costlier reference setting.

The model also deploys an MTP-3 speculative decoding head that StepFun reports delivers roughly a 30% end-to-end throughput speedup. Training used FP8 mixed precision with reinforcement learning on what StepFun describes as million-level verifiable agentic environments. The FP8 checkpoint weighs 650.32 GB.

Hardware requirements and serving

Step 5 Preview is a data-center workload. MarkTechPost reports the minimum listed self-hosting configurations:

• FP8 via vLLM or SGLang: 4× B300 GPUs or 8× H200 GPUs, single node
• BF16 via vLLM: 8× B300 (single node) or 16× H200 (two nodes)
• Offline generation via Transformers: supported, though the full BF16 shard workflow is not yet validated end-to-end

vLLM and SGLang ship dedicated Docker images with OpenAI-compatible APIs. On Blackwell hardware, expert-parallel modes use the MegaMoE backend. StepFun's recommended inference settings are temperature 1.0, top_p 0.95, and top_k 0, with model-card results produced at the High reasoning setting.

Benchmark performance

All benchmark figures below are vendor-reported by StepFun unless otherwise noted. StepFun acknowledges that competitor results in its comparison chart used Max effort while Step 5 Preview ran at High reasoning, so the comparison is not like-for-like.

On StepFun's own evaluations, DeepSWE v1.1 scores 67.7, Terminal-Bench 4.0 reaches 33.3, and Agents' Last Exam CLI pass rate lands at 29.5. Independent figures come from Artificial Analysis via OpenRouter: 43.7 on the Intelligence Index, 88.3% on AA-LCR long-context reasoning, 46.5% on Humanity's Last Exam, and 54.3% on GDPval-AA. Vals AI surfaces weaker results: SRE Bench at 0.4% and Harvey's Legal Agent Benchmark at 9.2%.

StepFun's comparison chart places the model against GLM-5.3 and Kimi K3:

The StepFun API charges $0.10 per 1M tokens for cache hits, and output pricing includes reasoning tokens. At $1.00 input and $2.70 output, the API undercuts Kimi K3's $3.00/$15.00 substantially. As we've noted in our analysis of 1M-context pricing dynamics, extended-context cost structures increasingly determine adoption more than raw benchmark position.

AI Mastery analysis

The 27B active parameter figure is the number engineers should anchor on for deployment cost. Per-token FLOP cost is closer to a 27B dense model than to a 600B one, which makes the $1.00/$2.70 API price credible and the 4× B300 minimum self-host configuration relevant. However, the minimum hardware bar still places this outside reach for teams without data-center GPU access, reinforcing that local AI remains a workload decision, not a capability question.

The Terminal-Bench 4.0 score of 33.3 versus GLM-5.3's 41.9 is a meaningful gap for terminal-native agents, and the 0.4% SRE Bench result is a serious limitation for infrastructure automation. The 88.3% AA-LCR score is compelling for document-heavy pipelines and is independently sourced from Artificial Analysis, while the agent-task numbers remain self-reported at a lower reasoning effort than competitors. Engineers evaluating this model for long-horizon agentic execution should run their own harness comparisons before treating DeepSWE parity with GLM-5.3 and Kimi K3 as established.

Step 5 Preview's release continues a pattern where open weights arrive weeks after a closed API launch. The 92-block MoE, Sparse GQA with 8× block compression, and speculative decoding speedup represent architectural specificity rather than a parameter-count story, and it positions the model as a credible option for long-context agent pipelines wherever data-center GPU access is available.

Sources

• Primary: stepfun-ai/Step-5-Preview · Hugging Face
• Reported by: MarkTechPost: StepFun Releases Step 5 Preview Open Weights: 600B MoE, 27B Active, 1M Context