Google open-sources TorchTPU, letting PyTorch models run natively on TPUs

Summary

• Google Cloud released TorchTPU as open source on October 20, 2026, at the PyTorch Conference.
• The library, built with Meta's PyTorch team, runs PyTorch workloads on TPUs with minimal code changes.
• It widens AI infrastructure options for teams currently tied to Nvidia or Amazon silicon.

The latest

PyTorch workloads can now run directly on Google's TPUs with minimal code rewriting, after Google Cloud open-sourced TorchTPU on October 20, 2026, at the PyTorch Conference 2026. The library was developed with Meta's PyTorch team and uses standard PyTorch APIs, allowing eager-mode debugging before switching to production performance through torch.compile. The code is published on GitHub.

Details

• The release: Google Cloud said TorchTPU is a library it built in collaboration with the PyTorch team at Meta, designed so existing PyTorch workloads move onto TPUs without being rewritten for a different framework. It is available at github.com/google-pytorch/torch_tpu.
• How it works: According to Google, developers keep the familiar PyTorch APIs, debug in eager mode where operations run step by step, then compile through torch.compile for production-level performance. The stated goal is minimal code change rather than a separate TPU-specific codebase.
• Hugging Face support: The library directly supports version 5 of Hugging Face's Transformers library, Google said, with models deployable to TPUs by passing device_map="tpu". That places the most widely used model-loading interface in open-source AI on TPU hardware by default.
• Serving engines: Google said TorchTPU adds native support for vLLM through Inferact, released as vllm-torchtpu, and for SGLang through RadixArk, released as sglang-torchtpu. Both are the dominant open-source engines for high-throughput large language model inference.
• The llm-d launch: Google announced llm-d 1.0 alongside Red Hat and IBM. The project sits in the CNCF Sandbox and counts more than 500 contributors across 25 organizations, according to Google, which also announced TPU support inside the Ray platform via Anyscale.
• Training numbers: Google reported that TorchTitan was validated on up to 8,000 seventh-generation Ironwood TPUs, reaching 52% model FLOPs utilization on the Qwen 3 8B model, with near-linear scaling tested out to 8,192 chips.
• Inference numbers: Google said Zenteiq recorded more than a fourfold increase in total inference throughput using GKE Inference Gateway with llm-d, lifting daily generated token capacity from 1.47 billion to 5.97 billion tokens across 1,000 TPU v6e chips.
• Partner ecosystem: The announcement leans on outside companies rather than Google alone: Inferact, RadixArk, Anyscale, Zenteiq, Red Hat and IBM each carry a named piece of the stack, from serving engines to orchestration to the deployment benchmark Google cites.

Background

PyTorch, originally developed at Meta, is the default framework for most AI research and model training, and the bulk of that work runs on Nvidia GPUs. TPUs are Google's in-house accelerators, historically most accessible through Google's own JAX and TensorFlow stacks.

Between the lines

The pattern across the announcement is compatibility rather than migration: Transformers, vLLM, SGLang and Ray are the tools teams already use, and Google is attaching TPU support to each instead of asking developers to adopt a Google-native framework. The performance claims Google published — 52% utilization at 8,000 chips, a fourfold throughput gain at Zenteiq — are the company's own figures, offered as evidence that the compatibility path does not cost performance at scale.

What's next

Watch for independent benchmarks of TorchTPU against GPU baselines, contribution activity on the GitHub repository, and whether llm-d 1.0 advances beyond CNCF Sandbox status as its 25-organization contributor base expands.

Source: Google Cloud