Amazon takes aim at Nvidia with TorchNeuron, loosening PyTorch's grip on GPUs

On Monday, Amazon Web Services unveiled a tool called TorchNeuron that lets AI startups move their applications onto Amazon's in-house chip, Trainium. In doing so, the company is taking a direct swing at the software advantage that has made Nvidia the default choice for AI developers.

TorchNeuron works with PyTorch, the free, open-source toolkit used to build, train, and run AI models — and, by Amazon's count, the framework behind roughly 85% of all AI apps. PyTorch was originally created at Facebook and has been stewarded by the Linux Foundation since 2022. Amazon says a PyTorch app can be made to run on Trainium by changing a single line of configuration, with no code rewrite required.

Why Nvidia sits at the center

Nvidia has spent years making sure its CUDA software supports PyTorch seamlessly, and as a result its hardware is now baked into the framework almost by default. TorchNeuron is adapted from PyTorch, so developers who already know the toolkit need no retraining. Amazon isn't the first to offer such a path, though: PyTorch users can already port their apps to AMD GPUs or even Apple's chips with a simple configuration change.

Amazon's real pitch lies elsewhere: moving to Trainium opens the door to the entire AWS cloud. The company contrasts itself with so-called neoclouds such as CoreWeave, Nebius, and Firmus, many of which are tied to Nvidia hardware. "You're building on AWS. You're not betting on a single-purpose compute provider," Colin Brace, VP of Technology Software at AWS Annapurna Labs, told TechCrunch.

Economics and a single platform

AWS argues that because the same chip handles both training and inference, it improves the economics of a full workload — even if its per-token price isn't the cheapest. Brace pointed to Deepgram, a voice-AI provider competing with ElevenLabs, which he said now processes 70% more audio per dollar after moving to Trainium.

But Amazon isn't stopping at an easy migration into its own cloud. The company says it intends to make PyTorch itself less automatically centered on Nvidia, and will contribute the necessary code upstream — stripping out parts of the framework that tie it to specific chips. The goal, it says, is to make PyTorch hardware-agnostic for the whole community, not just Trainium users.

The tactic is plainly good for anyone selling hardware and cloud services that compete with Nvidia. Yet part of PyTorch's whole point is to hide the messy business of getting AI code to run on hardware, and it remains to be seen whether pushing Nvidia aside delivers the choice and lower costs Amazon promises — or just more configuration headaches. Either way, if Amazon truly decouples PyTorch from Nvidia, the consequences for the GPU giant could run well beyond ordinary chip competition. "The real story is removing the lock-in that kept developers from having a real choice at all," Brace said.

Source: TechCrunch