Nvidia’s first benchmark outing for its Vera Rubin NVL72 rack landed on Wednesday in the preview category of MLPerf Inference v6.1, and the company is claiming a wide gap over the hardware it replaces.
Against a GB300 NVL72, the new rack reported up to 3.7x the throughput on Qwen3-VL. The figure spans three scenarios, offline, server and interactive, and came out of vLLM running with Nvidia’s Dynamo framework. On DeepSeek-R1, served through TensorRT-LLM, the gain was up to 2.5x. Both entries sit in the Closed division, where every system runs the same reference model so results stay comparable.
Two design choices explain the numbers. Inside the rack, Tensor Cores and a Transformer Engine carry both stages of a request: the prefill pass that reads the prompt and the decode pass that produces the answer. A narrower number format, NVFP4, keeps weights, attention state and the KV cache small enough to move quickly. Submissions also split prefill from decode across separate pools of hardware and spread mixture-of-experts layers across many experts at once.
Scaling held up. One submission grew from 72 GPUs in a single GB300 rack to 288 across four racks while keeping 99 percent efficiency in the offline scenario. A text-to-video test on GB300 produced 0.65 clips per second at 5.7 seconds each.
Agentic work earned its own line: 30x better than GB300 in preview testing on the SemiAnalysis AgentX benchmark, with MLCommons preparing an Endpoints benchmark for that workload class. Nebius filed its own Vera Rubin entries as part of a round with 19 participating partners.
