Meta Runs Google TPUs in Production via New PyTorch Integration

Google’s custom Tensor Processing Units (TPUs) are now operating in production environments at Meta, a move that signifies the first time the hardware has been sold to external customers for such workloads. This development was confirmed through the release of a new PyTorch-native version of the SGLang inference engine, which allows Meta’s teams to run AI models on Google silicon without extensive code rewrites. The integration represents a strategic pivot for Meta, which previously relied heavily on Nvidia’s graphics processing units for its massive data center operations.

The technical breakthrough centers on a backend known as TorchTPU, which Google co-developed with Meta’s PyTorch engineering team. Zongwei Zhou, Meta’s lead for PyTorch-TPU at RadixArk, stated that the SGLang-TorchTPU system is actively running in production. He described the result as a "remarkably mature, PyTorch-native serving stack," achieved through a close three-way collaboration between Meta, Google, and the SGLang team. RadixArk, a startup founded by the creators of SGLang, built the engine in partnership with the two tech giants.

Jana van Greunen, director of PyTorch engineering at Meta, emphasized the strategic importance of hardware diversity. She noted that Meta "cannot be constrained by a single hardware architecture" and that the new backend "enables us to deploy our PyTorch workloads to Google TPUs." This move follows years of friction, as Meta previously used an older bridge called PyTorch/XLA to access TPUs between 2020 and 2023 before canceling those contracts. The new solution is designed specifically for teams that prefer to stay within the PyTorch ecosystem rather than migrating to Google’s JAX framework.

Performance metrics suggest the new integration is competitive with Nvidia’s current offerings. Inferact, a startup co-founded by the creators of the vLLM engine, reported that a tuned vLLM setup on the seventh-generation Ironwood TPU achieved over 700 tokens per second for a single user. This test involved serving Moonshot AI’s open-weight Kimi K3 model, which has 2.8 trillion parameters. The benchmark compared 16 Ironwood chips using speculative decoding against 16 Nvidia GB200s, which recorded 452 tokens per second. Without speculative decoding, the new kernels delivered throughput 1.4 to 2 times higher than the GB200 baseline at small batch sizes.

The announcement lands just one day after Amazon revealed it was open-sourcing TorchNeuron, a similar native PyTorch backend for its Trainium2 and Trainium3 chips. Both developments appear calculated to loosen Nvidia’s software lock-in, which has long dominated the AI hardware market. Meta reportedly signed a multi-year rental agreement for Google Cloud TPUs, with talks last year covering rentals for 2026 and potential purchases for 2027. While the exact location of Meta’s production TPUs remains unclear, the shift underscores a growing trend among major tech firms to diversify their hardware suppliers.

Looking ahead, the SGLang-TorchTPU project plans to expand model support, starting with DeepSeek V4.1 Flash and Reflection Beam. RadixArk aims to provide Day 0 TPU support for new open models by the first quarter of 2027. Google, Meta, and RadixArk will share further details at the SGLang Summit 2026, scheduled for November 12–13 in San Francisco. This ongoing collaboration highlights the increasing complexity and interdependence of the AI infrastructure ecosystem, where software compatibility is becoming as critical as raw hardware speed.