Baidu official announcement: Wenxinyiyan is about to come out, and it may be stronger than ChatGPT!

​Author | Xu Jiecheng

On February 7, Baidu’s official public account released a very brief announcement - "Official Announcement: A Word from Wen Xin". It is understood that Wen Xinyiyan is a ChatGPT-like chatbot developed internally by Baidu, and its English name is ERNIE Bot.

According to Baidu insiders: Wen Xinyiyan is extended based on the knowledge enhancement large model (Ernie) proposed by Baidu It is composed of a series of advanced large models that can perform a wide range of tasks, including language understanding, language generation (ERNIE 3.0 Titan), and image generation from text (ERNIE-ViLG). Compared with other language models, Wenxinyiyan can combine extensive knowledge with massive data to produce extraordinary understanding and generation capabilities. The company plans to complete internal testing of Wen Xinyiyan in March and then officially open it to the public.

Although we don’t know Wen Xinyiyan’s actual performance for the time being, through Baidu’s previous release of ERNIE 3.0 Titan: Exploring Larger-scale Knowledge Enhanced Pre-training for Language Understanding and Generation" paper, we can currently get some general information about the language model ERNIE 3.0 Titan it applies to.

According to the paper: ERNIE 3.0 Titan is a 100-billion-parameter model trained by Baidu on the PaddlePaddle platform, which contains up to 260 billion parameters (ChatGPT uses GPT3 with 175 billion parameters. 5 models). In addition, Baidu also designed a self-supervised adversarial loss and a controllable language modeling loss to enable ERNIE 3.0 Titan to generate trustworthy and controllable text.

In order to reduce computing overhead and carbon emissions, Baidu also proposed an online distillation framework for ERNIE 3.0 Titan, in which the teacher model will simultaneously teach students and train itself. ERNIE 3.0Titan is the largest Chinese intensive pre-training model to date. Relevant experimental results show that the performance of ERNIE 3.0 Titan on 68 NLP data sets is better than the most advanced language models at this stage, including the GPT3.5 model applied by ChatGPT.

ERNIE 3.0 Titan model architecture diagram

It is reported that Wen Xinyiyan is currently in the pre-launch stage In the final sprint stage, the exposure of relevant news also caused Baidu's Hong Kong stock price to soar by more than 17%, and its market value increased by approximately HK$70 billion. According to speculation by some industry insiders, Baidu Wenxin Yiyan project may have started research and development as early as September 2022. At that time, Baidu CEO Robin Li said at the World Artificial Intelligence Conference: No matter at the technical level or the commercial application level, artificial intelligence There has been tremendous progress, and some have even changed direction.

Considering the current popularity of generative AI brought about by ChatGPT, Baidu is bound to be more than just domestic companies coveting the market in this field in the future. From a market perspective, the most obvious value and implementation scenario of ChatGPT-like chatbots is Baidu’s main search business. From this point of view, if the next technological revolution really breaks out in this field, then Baidu will undoubtedly take the lead. In addition, many people in the industry believe that considering the pace of advancement by Google and Microsoft, the progress of Wen Xinyiyan’s open internal testing may continue to advance.

The above is the detailed content of Baidu official announcement: Wenxinyiyan is about to come out, and it may be stronger than ChatGPT!. For more information, please follow other related articles on the PHP Chinese website!