Hugging Face models now run natively on Google TPUs
IA en un minuto newsroom · Editor: Jon Elgezabal
In 30 seconds
Hugging Face has made Transformers, Diffusers and Kernels run directly on TPUs, Google's AI processors, thanks to TorchTPU. Sending the model to that device is enough to use it, without the intermediate layer that used to be needed. The integration still has pending bugs, and Google has at the same time launched vLLM and SGLang on the same foundation.
On Tuesday, October 20, Hugging Face, the platform where AI models are shared, announced that its main libraries now run natively on Google's TPUs. TPUs are the chips Google has spent about ten years designing for artificial intelligence and relies on to train and run its own models, Gemini among them.
This is made possible by TorchTPU, which adds TPUs as one more device in PyTorch (a framework for programming AI models) and installs with a single command on a virtual machine with a TPU. Until now, using PyTorch on these chips meant going through PyTorch/XLA, an intermediate layer that, according to Hugging Face, worked but had its quirks: code did not always execute when expected, a small change could force a recompilation and bugs were hard to track down. The other way out was learning JAX, another programming framework.
In practice, all it takes is picking a Hugging Face model and sending it to the TPU, without touching the rest of the code. It works with Transformers, its main model library; with Diffusers, the one for images and video, which can also split a model too large for one chip across several; and with Kernels, optimized pieces of code that speed models up. With one of those kernels, on a TPU v6e-8 and with batches of 64 requests, text generation is 21.5 times faster than with the default implementation on the Qwen3-30B-A3B model, and 12.8 times faster on Gemma 4 26B-A4B.
It is still a work in progress: 96.5% of the Transformers tests executed on TPU pass. Hugging Face has reported 34 bugs to the TorchTPU team, 19 of which are already fixed, and the tests also turned up errors in Transformers itself, which have since been corrected. In addition, Optimum TPU, the tool Hugging Face built to use its models on these chips, is no longer needed to run them with Transformers and Diffusers.
Google presented, that same day, versions of vLLM and SGLang, two programs for running AI models and handling requests, that work on top of TorchTPU. According to its developer blog, the vLLM version was built with Inferact and the SGLang one with RadixArk; both already ran on TPUs, but through JAX. TPUs can be used on Google Cloud, from a single chip up to groups of thousands.
Why it matters · analysis and opinion
The barrier with TPUs was never the chip but the software: they grew up around TensorFlow and JAX, and anyone working in PyTorch had to adapt to an intermediate layer with its own rules. If Hugging Face's libraries now treat those chips as one more device, switching compute providers no longer means redoing the code, and Google gains an argument to attract teams that until now did not even consider its chips. For anyone training or serving open models in the cloud, the choice comes down to price and performance with the same program. What is still missing is maturity: Hugging Face itself admits that some tests fail and some bugs remain open. Before moving a production service, the prudent step is to measure first with the specific model that will be used.
Official source: Hugging Face · Written with the help of AI: how we make the news
