Google open-sourced TorchTPU at PyTorch Conference 2026 on Tuesday, a backend that makes its tensor processors a native device inside the framework rather than a target reached through a translation layer.
Meta’s PyTorch team built it jointly with Google, which now recommends it for all new workloads and bills it as the successor to PyTorch/XLA.
The change is structural. Under XLA, developers worked with lazy tensor graphs, had to mark compilable regions themselves, and got pure SPMD only. PrivateUse1 is the route TorchTPU took instead, giving ordinary tensors, no subclasses, one process per device, preserved views and in-place mutations. Divergent MPMD execution is supported.
Four repositories, one runtime
The release is split across TorchTitan, SGLang-TorchTPU, vLLM-TorchTPU and the core TorchTPU package, all four open. Version 0.2.0 is the current PyPI release, Python 3.11 is the minimum runtime, and the licence is Apache 2.0.
Testing reached from v5e parts all the way to 8,192 Ironwood chips in one cluster. Checkpoints in FP8, MXFP4 and NVFP4 can be served.
Training Qwen 3 8B with TorchTitan on Ironwood hit 52% model flop utilisation by the company’s own count. A fine-tuning case started out of box in eager mode at 11.49 seconds per step and reached 2.78 seconds once fully tuned.
Two dispatch modes sit behind the interface. Each operator runs asynchronously under Strict, with standard semantics. DeferAndFuse instead fuses runs of eager operators to cut memory traffic, and the accompanying paper puts the speedup over Strict at 50% or more.
The strategic read is competitive. PyTorch is where most AI code is written, and Google has now removed the main reason teams cite for staying on Nvidia hardware when they would rather not.
