Depth over width is StepFun’s bet. The Chinese lab previewed Step 5, a sparse mixture-of-experts model that stacks 92 Transformer layers in a deliberately narrow shape, holding roughly 600B total parameters while activating about 27B of them per token.
The case for that geometry is agentic work. Deeper stacks give implicit multi-hop reasoning more room during the long prefill phases agents spend searching, running code and reading tool output. Training used long-horizon, on-policy reinforcement learning with bit-wise alignment between training and inference, alongside speculative decoding, FP8 experts and key-value cache offload. StepFun reports better than 3x end-to-end speedup on long tasks.
Its published results trail the leaders. StepCodeBench gave 49.0, DeepSWE v1.1 gave 67.7 and ProgramBench 80.5, all below GPT-6 Astra and Claude Opus 5. Two 24-hour experiments went better: tuning an H100 kernel to 508 TFLOPS against Opus 5’s 493, and lifting Qwen3-30B-A3B on AIME24 from 53.3 to 60 percent.
Independent scoring from Artificial Analysis puts Step 5 at 44 on its Intelligence Index, where the median reasoning model in that price band sits at 24. It also burned 160M output tokens against a 92M median, so verbosity nibbles at the savings. Open weights arrive October 15.
