NVIDIA releases H200, the world’s most powerful AI chip: performance nearly doubled compared to H100

NVIDIA releases the world’s most powerful AI chip H200: performance nearly doubled compared to H100

Quick Technology reported today (November 14) that at the 2023 Global Supercomputing Conference (SC2023), chip giant Nvidia released the successor to the H100 chip, which is currently the world’s most powerful AI chip-H200.

The performance of H200 is directly improved by 60% to 90% compared to H100

Not only that, H200 and H100 are both based on the NVIDIA Hopper architecture, which also means that the two chips are compatible with each other. For companies using H100, they can be seamlessly replaced with the latest H200.

H200 is Nvidia’s first chip to use HBM3e memory. It has faster speed and larger capacity, and is very suitable for training and inference of large language models

In addition to HBM3e memory, the memory capacity of H200 is 141GB, and the bandwidth has increased from 3.35TB/s of H100 to 4.8TB/s.

The performance improvement of H200 is mainly reflected in the inference performance of large models. The inference speed of H200 on the large Llama2 model with 70 billion parameters is twice as fast as that of H100, and the inference energy consumption of H200 is directly reduced compared to H100. half.

H200 has higher memory bandwidth, which means that for memory-intensive high-performance computing applications, operating data can be accessed more efficiently. Using the H200 can increase result acquisition time by up to 110 times compared to a central processing unit (CPU)

Nvidia said that the H200 is expected to be shipped in the second quarter of 2024, and the price has not been announced yet. However, in the face of computing power shortage, big technology companies are still expected to stock up crazily.

The above is the detailed content of NVIDIA releases H200, the world’s most powerful AI chip: performance nearly doubled compared to H100. For more information, please follow other related articles on the PHP Chinese website!