Inference has a scheduling problem that adding chips does not fix. Cerebras Systems says it can get more work from the same silicon by splitting a request’s two phases across separate hardware pools, a pattern it calls disaggregation.
The two phases behave differently. Prefill chews through an entire prompt as one large matrix operation. Decode then emits tokens one at a time, reusing a cache of earlier computations. Put them on the same chip and the compute-heavy half can stall requests that are already streaming.
Separate the pools and operators can size, batch and schedule each half for its own goal, whether that is a fast first token or smooth streaming. The cost is a handoff. The cache built during prefill has to move to the decode pool before generation can continue.
Cerebras put the throughput gain at 5x in early tests with the same wafer footprint, and credits its on-chip SRAM. The company lists 21,000 TB/s of memory bandwidth per WSE-3 wafer, against 22 to 23 TB/s for HBM4 GPUs, though it cautions the figures describe different memory tiers rather than measured token speeds.
On GPT-oss-120B, Artificial Analysis data from September 10 puts Cerebras at 1,669 output tokens per second, ahead of SambaNova at 708 and Groq at 475. The company says it is now pursuing heterogeneous disaggregation, pairing partner accelerators for prefill with its own chips for decode.
