Google and Meta Open-Source TorchTPU, a Native PyTorch TPU Backend
By @sharedot · · 6 pages
- AI Frontier
- Open Source
- Pytorch
- Tpu
Google and Meta open-sourced TorchTPU on Oct 20, 2026, giving PyTorch developers a native TPU backend with minimal code changes.
What shipped: four repos, one native backend
At PyTorch Conference North America 2026 in San Jose on October 20–21, Google and Meta announced TorchTPU, an open-source, native PyTorch backend for Google's Tensor Processing Units.
Rather than a single library, the teams released four Apache 2.0-licensed repositories: TorchTPU (the core backend), vLLM-TorchTPU and SGLang-TorchTPU (inference), and TorchTitan (large-scale training covering dense, mixture-of-experts and diffusion workloads).
Bill Jia of Google Cloud and Meta software engineer Zongwei Zhou presented the project on stage, and the SGLang project confirmed it was open-sourcing TorchTPU for native, hardware-accelerated PyTorch execution on Cloud TPUs.
The effort began in mid-2025 in collaboration with Meta's PyTorch team, giving it over a year of runway before release.
Why the Meta–Google pairing is a reversal
A native TPU path for PyTorch erodes one of CUDA's practical moats: teams that never justified a rewrite into JAX can now point existing workloads at a TPU rental.
It is also a distribution play in a market where tight accelerator supply matters — Shattered notes Google's own CEO has said compute stays tight into 2027, and widening the developer pool for existing silicon changes the cloud accelerator calculus.
The evidence: benchmarks, wheels and CI
The torch-tpu package landed on PyPI as version 0.2.0 on October 19, 2026, with Linux x86-64 wheels for Python 3.11–3.14. Google Developers reports that Llama 3 405B reached 57% MFU on 1,024 TPU v7x (Ironwood) chips — a vendor-reported figure, not an independent benchmark.
The TorchTPU team says it evaluated bounded dynamism correctness on 839 causal LLMs from the 5K corpus and confirmed correct numerics on 834, about 99%, and that its CI/CD is now integrated into the official PyTorch CI HUD.
Google says the backend integrates with torch.compile, supports eager execution with optimizations, and lists Hugging Face Transformers, llm-d, Ray, TorchRec and LiteRT as tools with TPU routes.
Stakes and the six-to-twelve-month roadmap
Google Cloud says TorchTPU is already production-used for inference and training by some large customers while remaining in active development — the honest caveat being that real models with custom operators will surface a long tail of unsupported operations, and no support matrix maps features to TPU generations v5e, v5p, v6e or Ironwood.
The stated priorities are production hardening, precompiled dynamic ATen kernels, productionizing XLA bounded dynamism, GKE packaging, and FP8/FP4 quantization work for next-generation TPUv8 architectures.
For open-weight model builders, the practical effect is immediate: anyone serving with vLLM or SGLang can now keep their serving stack and swap the hardware underneath — check the roadmap before depending on dynamic shapes or quantization for your workload.