NVIDIA issued an article introducing 'Huang's Law': GPU inference performance increased by 1,000 times in ten years, without fear of the impact of Moore's Law

News from this site on October 1, just as people are debating whether Moore's Law will slow down in the 2020s, whether it is still applicable, or even dead, NVIDIA issued a document announcing "黄's Law(Huang's Law)".

According to a latest NVIDIA blog, In the past ten years, NVIDIA GPU artificial intelligence processing power has increased 1,000 times. "Huang's Law" means that the speedup seen in "single-chip inference performance" will not fade away but will continue to occur.

#According to NVIDIA chief scientist Bill Daly, "the tectonic shift in how computer performance will be achieved in the post-Moore's Law era" will depend primarily on humans. wisdom. Although this characteristic seems difficult to reduce to a law, Dali believes that the following chart marks the beginning of "Huang's Law"

this A search on the website found that "Huang's Law" is not Nvidia's own original statement. IEEE Spectrum was the first to propose a law named after Huang, and "Huang's Law" later became famous through a column in the Wall Street Journal.

According to Dally’s recent presentation at the Hot Chips 2023 conference, the graph above shows a 1000x increase in GPU AI inference performance over the past decade. What’s interesting, Dally said, is that unlike Moore’s Law, process shrinkage has had little impact on the progress of Huang’s Law.

Dally recalls how he achieved 16x growth by changing the underlying number processing of Nvidia GPUs. Another big boost came with the arrival of Nvidia's Hopper architecture using the Transformer engine. Hopper is said to achieve a 12.5x performance jump and energy savings using a dynamic combination of 8- and 16-bit floating point and integer math.

Dally also said that despite the diminishing benefits of Moore's Law, he believes that "Huang's Law" will continue to exist.

Advertising Statement: This article contains external jump links (including but not limited to hyperlinks, QR codes, passwords, etc.), which are intended to provide more information and save screening time. The results are for reference only. Please note that all articles on this site contain this statement

The above is the detailed content of NVIDIA issued an article introducing 'Huang's Law': GPU inference performance increased by 1,000 times in ten years, without fear of the impact of Moore's Law. For more information, please follow other related articles on the PHP Chinese website!