search
HomeTechnology peripheralsAINVIDIA slaps AMD in the face: With software support, H100's AI performance is 47% faster than MI300X!

NVIDIA slaps AMD in the face: With software support, H100s AI performance is 47% faster than MI300X!

According to news on December 14, AMD launched its most powerful AI chip Instinct MI300X earlier this month. The AI ​​performance of its 8-GPU server is 60% higher than that of Nvidia H100 8-GPU. In this regard, NVIDIA recently released a set of the latest performance comparison data between H100 and MI300X, showing how H100 can use the right software to provide faster AI performance than MI300X.

According to data previously released by AMD, the FP8/FP16 performance of MI300X has reached 1.3 times that of NVIDIA H100, and the speed of running Llama 2 70B and FlashAttention 2 models is 20% faster than H100. In the 8v8 server, when running the Llama 2 70B model, MI300X is 40% faster than H100; when running the Bloom 176B model, MI300X is 60% faster than H100.

However, it should be noted that when comparing MI300X with NVIDIA H100, AMD used the optimization library in the latest ROCm 6.0 suite (which can support the latest computing formats such as FP16, Bf16 and FP8, including Sparsity etc.) to get these numbers. In contrast, the NVIDIA H100 was not tested without the use of optimization software such as NVIDIA's TensorRT-LLM.

AMD's implicit statement on the NVIDIA H100 test shows that using vLLM v.02.2.2 inference software and the NVIDIA DGX H100 system, the Llama 2 70B query has an input sequence length of 2048 and an output sequence length of 128

The latest test results released by NVIDIA for DGX H100 (with 8 NVIDIA H100 Tensor Core GPUs, with 80 GB HBM3) show that the public NVIDIA TensorRT LLM software is used, of which v0.5.0 is used for Batch-1 Test, v0.6.1 for latency threshold measurements. Test workload details are the same as previously conducted AMD tests

NVIDIA slaps AMD in the face: With software support, H100s AI performance is 47% faster than MI300X!

According to the results, after using optimized software, the performance of the NVIDIA DGX H100 server has increased by more than 2 times, and is 47% faster than the MI300X 8-GPU server displayed by AMD

DGX H100 can handle a single inference task in 1.7 seconds. In order to optimize response time and data center throughput, cloud services set fixed response times for specific services. This allows them to combine multiple inference requests into larger "batches", thereby increasing the overall number of inferences per second on the server. Industry standard benchmarks such as MLPerf also use this fixed response time metric to measure performance

Slight tradeoffs in response time can create uncertainty in the number of inference requests the server can handle in real time. Using a fixed 2.5 second response time budget, the NVIDIA DGX H100 server can handle more than 5 Llama 2 70B inferences per second, while Batch-1 handles less than one per second.

Obviously, it is relatively fair for Nvidia to use these new benchmarks. After all, AMD also uses its optimized software to evaluate the performance of its GPUs, so why not do the same when testing the Nvidia H100?

You must know that NVIDIA's software stack revolves around the CUDA ecosystem, and after years of hard work and development, it has a very strong position in the artificial intelligence market, while AMD's ROCm 6.0 is new and has not yet been tested in real-world scenarios.

According to information previously disclosed by AMD, it has reached a large part of the deal with large companies such as Microsoft and Meta. These companies regard its MI300X GPU as a replacement for Nvidia's H100 solution.

AMD’s latest Instinct MI300X is expected to be shipped in large quantities in the first half of 2024. However, NVIDIA’s more powerful H200 GPU will also be shipped by then, and NVIDIA will also launch a new generation of Blackwell B100 in the second half of 2024. In addition, Intel will also launch its new generation AI chip Gaudi 3. Next, competition in the field of artificial intelligence seems to become more intense.

Editor: Xinzhixun-Rurounijian

The above is the detailed content of NVIDIA slaps AMD in the face: With software support, H100's AI performance is 47% faster than MI300X!. For more information, please follow other related articles on the PHP Chinese website!

Statement
This article is reproduced at:搜狐. If there is any infringement, please contact admin@php.cn delete
Gemma Scope: Google's Microscope for Peering into AI's Thought ProcessGemma Scope: Google's Microscope for Peering into AI's Thought ProcessApr 17, 2025 am 11:55 AM

Exploring the Inner Workings of Language Models with Gemma Scope Understanding the complexities of AI language models is a significant challenge. Google's release of Gemma Scope, a comprehensive toolkit, offers researchers a powerful way to delve in

Who Is a Business Intelligence Analyst and How To Become One?Who Is a Business Intelligence Analyst and How To Become One?Apr 17, 2025 am 11:44 AM

Unlocking Business Success: A Guide to Becoming a Business Intelligence Analyst Imagine transforming raw data into actionable insights that drive organizational growth. This is the power of a Business Intelligence (BI) Analyst – a crucial role in gu

How to Add a Column in SQL? - Analytics VidhyaHow to Add a Column in SQL? - Analytics VidhyaApr 17, 2025 am 11:43 AM

SQL's ALTER TABLE Statement: Dynamically Adding Columns to Your Database In data management, SQL's adaptability is crucial. Need to adjust your database structure on the fly? The ALTER TABLE statement is your solution. This guide details adding colu

Business Analyst vs. Data AnalystBusiness Analyst vs. Data AnalystApr 17, 2025 am 11:38 AM

Introduction Imagine a bustling office where two professionals collaborate on a critical project. The business analyst focuses on the company's objectives, identifying areas for improvement, and ensuring strategic alignment with market trends. Simu

What are COUNT and COUNTA in Excel? - Analytics VidhyaWhat are COUNT and COUNTA in Excel? - Analytics VidhyaApr 17, 2025 am 11:34 AM

Excel data counting and analysis: detailed explanation of COUNT and COUNTA functions Accurate data counting and analysis are critical in Excel, especially when working with large data sets. Excel provides a variety of functions to achieve this, with the COUNT and COUNTA functions being key tools for counting the number of cells under different conditions. Although both functions are used to count cells, their design targets are targeted at different data types. Let's dig into the specific details of COUNT and COUNTA functions, highlight their unique features and differences, and learn how to apply them in data analysis. Overview of key points Understand COUNT and COU

Chrome is Here With AI: Experiencing Something New Everyday!!Chrome is Here With AI: Experiencing Something New Everyday!!Apr 17, 2025 am 11:29 AM

Google Chrome's AI Revolution: A Personalized and Efficient Browsing Experience Artificial Intelligence (AI) is rapidly transforming our daily lives, and Google Chrome is leading the charge in the web browsing arena. This article explores the exciti

AI's Human Side: Wellbeing And The Quadruple Bottom LineAI's Human Side: Wellbeing And The Quadruple Bottom LineApr 17, 2025 am 11:28 AM

Reimagining Impact: The Quadruple Bottom Line For too long, the conversation has been dominated by a narrow view of AI’s impact, primarily focused on the bottom line of profit. However, a more holistic approach recognizes the interconnectedness of bu

5 Game-Changing Quantum Computing Use Cases You Should Know About5 Game-Changing Quantum Computing Use Cases You Should Know AboutApr 17, 2025 am 11:24 AM

Things are moving steadily towards that point. The investment pouring into quantum service providers and startups shows that industry understands its significance. And a growing number of real-world use cases are emerging to demonstrate its value out

See all articles

Hot AI Tools

Undresser.AI Undress

Undresser.AI Undress

AI-powered app for creating realistic nude photos

AI Clothes Remover

AI Clothes Remover

Online AI tool for removing clothes from photos.

Undress AI Tool

Undress AI Tool

Undress images for free

Clothoff.io

Clothoff.io

AI clothes remover

AI Hentai Generator

AI Hentai Generator

Generate AI Hentai for free.

Hot Article

R.E.P.O. Energy Crystals Explained and What They Do (Yellow Crystal)
1 months agoBy尊渡假赌尊渡假赌尊渡假赌
R.E.P.O. Best Graphic Settings
1 months agoBy尊渡假赌尊渡假赌尊渡假赌
Will R.E.P.O. Have Crossplay?
1 months agoBy尊渡假赌尊渡假赌尊渡假赌

Hot Tools

MinGW - Minimalist GNU for Windows

MinGW - Minimalist GNU for Windows

This project is in the process of being migrated to osdn.net/projects/mingw, you can continue to follow us there. MinGW: A native Windows port of the GNU Compiler Collection (GCC), freely distributable import libraries and header files for building native Windows applications; includes extensions to the MSVC runtime to support C99 functionality. All MinGW software can run on 64-bit Windows platforms.

Notepad++7.3.1

Notepad++7.3.1

Easy-to-use and free code editor

WebStorm Mac version

WebStorm Mac version

Useful JavaScript development tools

Dreamweaver Mac version

Dreamweaver Mac version

Visual web development tools

SublimeText3 Mac version

SublimeText3 Mac version

God-level code editing software (SublimeText3)