Nvidia Corporation (NASDAQ:NVDA) has changed the landscape of artificial intelligence chips with the unveiling of the H200, a graphics processing unit designed for training large language models.
The H200 has 1.4x the memory bandwidth and 1.8x the memory capacity of the H100 chip, which OpenAI used to train CPT-4.
The new chip will be able to handle intensive AI model training due to its memory upgrades, which include being the first of its kind to use a faster memory spec known as HBM3e.
Now, tech companies, governments and more will all vie for the top-of-the-line chips, which are expected to be released in the second quarter of 2024.
What remains to be seen is how many h200 chips Nvidia will produce and how much they will cost.
For a price point reference, H100 chips cost between $25,000 and $40,000, according to a Raymond James estimate, and the biggest AI models rely on thousands of them for training.
Notably, H200 chips will be compatible with systems that support H100s, meaning cloud providers won't need to make any changes as they introduce the newer model.
“The integration of faster and more extensive HBM memory serves to accelerate performance across computationally demanding tasks including generative AI models and [high-performance computing] applications while optimizing GPU utilization and efficiency,” Ian Buck, Nvidia’s VP of high-performance computing products, said in a video presentation this morning.
Contact Andrew Kessel at andrew.kessel@proactiveinvestors.com
Follow him on Twitter @andrew_kessel