Playing Catch Up: Amazon Enters AI Race with Custom Chips

Tech giant Amazon was one of the last big tech companies to join the generative AI gold rush, announcing its own Titan large language model in April, after Google announced Bard and Facebook launched LLaMA, following the massive success of OpenAI’s ChatGPT last year.

Now Amazon is aiming to catch up by creating custom microchips, the expensive and much sought-after hardware that powers generative AI models.

Via CNBC:

In an unmarked office building in Austin, Texas, two small rooms contain a handful of Amazon employees designing two types of microchips for training and accelerating generative AI. These custom chips, Inferentia and Trainium, offer AWS customers an alternative to training their large language models on Nvidia GPUs, which have been getting difficult and expensive to procure.

“The entire world would like more chips for doing generative AI, whether that’s GPUs or whether that’s Amazon’s own chips that we’re designing,” Amazon Web Services CEO Adam Selipsky told CNBC in an interview in June. “I think that we’re in a better position than anybody else on Earth to supply the capacity that our customers collectively are going to want.”

Amazon hopes that its investments in custom silicon production will give it a headstart in designing chips optimized for AI.

AWS quietly started production of custom silicon back in 2013 with a piece of specialized hardware called Nitro. It’s now the highest-volume AWS chip. Amazon told CNBC there is at least one in every AWS server, with a total of more than 20 million in use.

Currently, most companies deploying generative AI products rely on market leader Nvidia’s A100 GPUs, which cost roughly $10,000 per chip. The boom in large language models and other AI products has propelled Nvidia into the club of $1tn companies.

Allum Bokhari is the senior technology correspondent at Breitbart News. He is the author of #DELETED: Big Tech’s Battle to Erase the Trump Movement and Steal The Election. Follow him on Twitter @AllumBokhari.