search
HomeTechnology peripheralsAIThe latest news! Baidu Wenxin Big Model 4.0: The largest parameter model in the history of Wanka training, see you as soon as next week

The latest news! Baidu Wenxin Big Model 4.0: The largest parameter model in the history of Wanka training, see you as soon as next week


Yesterday, Cailian News exclusively revealed that Baidu’s Wenxin Model 4.0 is intensifying training and is close to being ready for release. Everyone has always been curious about Wen Xinyiyan's information. Today we also got more news about Wenxin 4.0, which involves key information such as underlying architecture, infrastructure, training data sets, costs, etc. It has a very high degree of credibility!
Let’s talk about the core conclusions first:
1. Yesterday’s revelations are basically true. It is currently understood that Wenxin Large Model 4.0 has actually been tested with small traffic.
2. The number of parameters of Wenxin 4.0 is larger than that of all LLMs with publicly released parameters. It is also the first large model in China to be trained using Wanka cluster.
3. The reasoning cost is much higher than that of Wenxin 3.5, it is said to be about 8-10 times! (Large models are really expensive!)
If these revelations are true, then this will be a major node for Baidu and even domestic large models to catch up with GPT-4.
Next, let’s take a look at the details of the revelations.
The largest parameter model in the history of Wanka cluster training?
According to the information we have received, the parameter scale of Wenxin Large Model 4.0 is larger than all LLMs currently publicly releasing parameters, which means that the parameter scale of Wenxin Large Model 4.0 is expected to exceed the trillion level.
Looking at this parameter amount alone, many people will think it's okay. After all, according to the currently revealed information, the parameter amount of GPT-4 is already around 1.8 trillion. However, the person who broke the news further stated that Wenxin Large Model 4.0 is still a single model and does not adopt the mixed expert model (MoE) used by GPT and many other large language models.
Previously, "genius hacker" George Hotez broke the news that the reason why GPT-4 uses a hybrid model is because the parameter size of the model cannot exceed 220 billion. OpenAI wants the model to get better, but if it just takes longer to train, the effect is already diminishing.
So, if Baidu can achieve a breakthrough in a single model, whether the model capabilities will also be significantly improved, we can only wait and see after the actual release.
A model with such a large number of parameters is bound to have high computing power requirements. The current news is that Wenxin 4.0 was trained on the Wanka AI cluster. It should be regarded as the first large language model in China to be trained using a Wanka-scale cluster.
What is the concept of Wanka cluster? In China, only Huawei and Alibaba have revealed that they have built Wanka AI cluster, but we have not seen a specific model based on it.
This shows that Wanka cluster is not easy to build, and it is even more difficult to use it to maximize its effect. According to analysis, it is precisely because of the deep integration of Fei Paddle that such a large-scale model can be efficiently trained based on the Wanka cluster.
The cost has surged, and low-traffic testing has been conducted for the public in a low-key manner
Not only is the training cost increasing, but the inference cost of Wenxin 4.0 has also been revealed to be much higher than that of 3.5. We have not yet obtained the specific inference cost per thousand tokens, but it is rumored that it was probably before 8-10 times, this is still in the case of high utilization (MFU). If utilization is even lower, costs are expected to continue to increase.
I have to say that large models are really expensive. Creating a leading underlying foundation model is a game for giants!
Finally, according to internal employees, Baidu has actually begun secretly testing Wenxin Big Model 4.0 with low traffic, and a small number of Wenxin Yiyan users are already using the latest model version.
Many people think this statement is more reliable, and we can also get some clues from some recent revelations in the technology community.
Perhaps, when you ask questions on Wenxin Yiyan now, you are using Wenxin Big Model 4.0. I don’t know if the generated results can compete with GPT-4.
I emphasize again that the above is not officially confirmed information, and everyone can judge its accuracy by themselves.

The above is the detailed content of The latest news! Baidu Wenxin Big Model 4.0: The largest parameter model in the history of Wanka training, see you as soon as next week. For more information, please follow other related articles on the PHP Chinese website!

Statement
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn
Understanding SQL WHERE ClauseUnderstanding SQL WHERE ClauseApr 11, 2025 am 09:07 AM

SQL WHERE Clause: A Comprehensive Guide The SQL WHERE clause is a fundamental component of SQL statements, used for filtering records and retrieving specific data from a database. Imagine a vast customer database – the WHERE clause allows you to pin

Newest Annual Compilation Of The Best Prompt Engineering TechniquesNewest Annual Compilation Of The Best Prompt Engineering TechniquesApr 10, 2025 am 11:22 AM

For those of you who might be new to my column, I broadly explore the latest advances in AI across the board, including topics such as embodied AI, AI reasoning, high-tech breakthroughs in AI, prompt engineering, training of AI, fielding of AI, AI re

Europe's AI Continent Action Plan: Gigafactories, Data Labs, And Green AIEurope's AI Continent Action Plan: Gigafactories, Data Labs, And Green AIApr 10, 2025 am 11:21 AM

Europe's ambitious AI Continent Action Plan aims to establish the EU as a global leader in artificial intelligence. A key element is the creation of a network of AI gigafactories, each housing around 100,000 advanced AI chips – four times the capaci

Is Microsoft's Straightforward Agent Story Enough To Create More Fans?Is Microsoft's Straightforward Agent Story Enough To Create More Fans?Apr 10, 2025 am 11:20 AM

Microsoft's Unified Approach to AI Agent Applications: A Clear Win for Businesses Microsoft's recent announcement regarding new AI agent capabilities impressed with its clear and unified presentation. Unlike many tech announcements bogged down in te

Selling AI Strategy To Employees: Shopify CEO's ManifestoSelling AI Strategy To Employees: Shopify CEO's ManifestoApr 10, 2025 am 11:19 AM

Shopify CEO Tobi Lütke's recent memo boldly declares AI proficiency a fundamental expectation for every employee, marking a significant cultural shift within the company. This isn't a fleeting trend; it's a new operational paradigm integrated into p

IBM Launches Z17 Mainframe With Full AI IntegrationIBM Launches Z17 Mainframe With Full AI IntegrationApr 10, 2025 am 11:18 AM

IBM's z17 Mainframe: Integrating AI for Enhanced Business Operations Last month, at IBM's New York headquarters, I received a preview of the z17's capabilities. Building on the z16's success (launched in 2022 and demonstrating sustained revenue grow

5 ChatGPT Prompts To Stop Depending On Others And Trust Yourself Fully5 ChatGPT Prompts To Stop Depending On Others And Trust Yourself FullyApr 10, 2025 am 11:17 AM

Unlock unshakeable confidence and eliminate the need for external validation! These five ChatGPT prompts will guide you towards complete self-reliance and a transformative shift in self-perception. Simply copy, paste, and customize the bracketed in

AI Is Dangerously Similar To Your MindAI Is Dangerously Similar To Your MindApr 10, 2025 am 11:16 AM

A recent [study] by Anthropic, an artificial intelligence security and research company, begins to reveal the truth about these complex processes, showing a complexity that is disturbingly similar to our own cognitive domain. Natural intelligence and artificial intelligence may be more similar than we think. Snooping inside: Anthropic Interpretability Study The new findings from the research conducted by Anthropic represent significant advances in the field of mechanistic interpretability, which aims to reverse engineer internal computing of AI—not just observe what AI does, but understand how it does it at the artificial neuron level. Imagine trying to understand the brain by drawing which neurons fire when someone sees a specific object or thinks about a specific idea. A

See all articles

Hot AI Tools

Undresser.AI Undress

Undresser.AI Undress

AI-powered app for creating realistic nude photos

AI Clothes Remover

AI Clothes Remover

Online AI tool for removing clothes from photos.

Undress AI Tool

Undress AI Tool

Undress images for free

Clothoff.io

Clothoff.io

AI clothes remover

AI Hentai Generator

AI Hentai Generator

Generate AI Hentai for free.

Hot Article

R.E.P.O. Energy Crystals Explained and What They Do (Yellow Crystal)
3 weeks agoBy尊渡假赌尊渡假赌尊渡假赌
R.E.P.O. Best Graphic Settings
3 weeks agoBy尊渡假赌尊渡假赌尊渡假赌
R.E.P.O. How to Fix Audio if You Can't Hear Anyone
3 weeks agoBy尊渡假赌尊渡假赌尊渡假赌
WWE 2K25: How To Unlock Everything In MyRise
3 weeks agoBy尊渡假赌尊渡假赌尊渡假赌

Hot Tools

MantisBT

MantisBT

Mantis is an easy-to-deploy web-based defect tracking tool designed to aid in product defect tracking. It requires PHP, MySQL and a web server. Check out our demo and hosting services.

ZendStudio 13.5.1 Mac

ZendStudio 13.5.1 Mac

Powerful PHP integrated development environment

SublimeText3 Chinese version

SublimeText3 Chinese version

Chinese version, very easy to use

PhpStorm Mac version

PhpStorm Mac version

The latest (2018.2.1) professional PHP integrated development tool

SecLists

SecLists

SecLists is the ultimate security tester's companion. It is a collection of various types of lists that are frequently used during security assessments, all in one place. SecLists helps make security testing more efficient and productive by conveniently providing all the lists a security tester might need. List types include usernames, passwords, URLs, fuzzing payloads, sensitive data patterns, web shells, and more. The tester can simply pull this repository onto a new test machine and he will have access to every type of list he needs.