Is it possible that Bing is integrating GPT-4? Is this true or is it a baseless theory?

​Author | Xu Jiecheng

Before the release of Microsoft New Bing, most people’s predictions were that the overall performance of New Bing would be very similar to ChatGPT. Even if there are some differences in details between the two, users should not be able to detect these differences from an experience perspective. However, with the release and update of New Bing, more and more people began to suspect that the previous $10 billion deal between Microsoft and OpenAI seemed to be related to more than just the "baby" ChatGPT.

1. Microsoft official announcement: Integrating a new model

Although New Bing and ChatGPT are both based on GPT The 3.5 series model was fine-tuned, but in mid-February, Microsoft announced that it had integrated a new AI language model into New Bing, allowing it to have better performance than ChatGPT. Although Microsoft has not clearly stated the source and related data of the new AI model to this day, some people in the industry speculate that the model is likely to be related to GPT-4, which OpenAI has not disclosed for a long time.

In fact, as early as the end of last year, a large number of rumors about the GPT-4 model had leaked out in the industry. Some news said that the GPT-4 model may have more than 100 trillion parameters (GPT -3 is 175 billion), and some experts predict that GPT-4 will pay more attention to cost control, and the number of model parameters will not change much compared to GPT-3. According to some foreign media reports at the time, GPT-4 would be released around February 2023. Although judging from OpenAI’s recent performance and its CEO’s speech, this information is not accurate. But according to the latest news from the New York Times, the GPT-4 model will likely be released in the first half of 2023.

And relevant test data also shows that the updated New Bing potential model has lower latency than ChatGPT. In addition, judging from the actual conversation performance, there are certain differences between the current New Bing and ChatGPT. During the conversation with users, New Bing shows a more natural and instinctive state than ChatGPT. But when the persistence of conversations increased, New Bing also showed a tendency of repeated responses like the GPT series models.

2. The only solution

During the release of ChatGPT, OpenAI once posted on its official blog that the company had "moved from their early models such as GPT-3 and Lessons learned from the deployment of Codex", we chose to use a hybrid model of RLHF (using human feedback signals to directly optimize language models) and supervised learning to train ChatGPT. According to OpenAI, using the RLHF method can reduce the probability of ChatGPT outputting malicious and erroneous information to a certain extent.

But from the perspective of New Bing, Microsoft at that time had to speed up the training cycle of New Bing due to competition with Google Bard. The less than three months between Microsoft's launch of the New Bing project and its release was not enough to rebuild and integrate the complete RLHF training channel. Therefore, Microsoft should still use the traditional supervised learning model when training New Bing's model. This has also caused New Bing to be plagued by accuracy problems caused by hasty development and lack of early training since its release.

According to relevant news, Microsoft is currently trying to solve the problem of accuracy of content generated by New Bing through various methods and has achieved some results, but the optimization plan does not explicitly mention it. and New Bing use the RLHF mode to retrain the model. In this regard, some experts speculate that Microsoft is likely to optimize the generated content of New Bing by integrating other AI language model generation filters.

Former Playment AI researcher Khamisi also analyzed this problem in his personal blog: "Neither ChatGPT nor other currently public models can well control the generation of malicious information and false information. , if Microsoft wants to lift the limit on the number of questions and answers on New Bing, the best way is to wait for OpenAI to provide some models with this capability and use them in New Bing. Now it seems that Microsoft has done this to some extent. thing."

3. The inseparable cooperation stage

In addition, another point worth noting is that although the cooperative relationship between OpenAI and Microsoft has always been very close, in Before "About ChatGPT Cooperation", the relationship between the two was far from reaching the level of "wearing the same pants". In previous collaborations and deals, they had never been open to each other when it came to data sets and their respective infrastructures.

However, just at the beginning of this year, Microsoft announced the third phase of its long-term partnership with OpenAI. In addition to billions of dollars of investment over several years, the more important point is: In this phase, Microsoft allows OpenAI companies and their developers to access Microsoft’s best internal AI infrastructure, models and tool chains through Azure. After that, OpenAI immediately further increased its investment in model data.

According to foreign media reports: Since the end of January, OpenAI has outsourced its data labeling work to a data company in San Francisco, trying to rely on more accurate data to build better content filters for some of its AI models. This move by OpenAI can basically be seen as a sincere expression of support for Microsoft after receiving support. Combining the above phenomena, we have reason to believe that Microsoft is likely to have obtained some rights to use the GPT-4 model and has begun to try to integrate it into its own search engine and conversational robots.

4. Written at the end

In any case, the cooperation between Microsoft and OpenAI has entered the deep water zone, and the public release of GPT-4 is just around the corner. If OpenAI wants to conduct a more complete test of GPT-4 before its official release, then big brother Microsoft, which has massive data and users, is undoubtedly their best choice, and Microsoft, which has made countless profits from ChatGPT, will most likely be happy to accept it. This young man’s request, after all, in today’s AI circle, nothing has a stronger ability to attract money than the GPT model.

But judging from previous speculations about the GPT-4 model, even if New Bing has integrated the GPT-4 model internally, it should only rely on the model to Some of New Bing's existing features have been optimized and the innovative capabilities of GPT-4 have not been made public. This move is indeed in line with Microsoft's conservative style, not to mention that its competitor Google just demonstrated it through Bard not long ago. What does "I can't eat hot tofu in a hurry" mean? Although neither Microsoft nor OpenAI has disclosed more news about New Bing and GPT-4 to the outside world so far, after learning about some of their current developments, I believe everyone’s expectations for the future have increased. .

Reference link:

https://blogs.microsoft.com/blog/microsoftandopenaiextendpartnership/

https://www.nytimes.com /technology/microsoft-ai-chatgpt-bing.html

https://time.com/6247678/openai-chatgpt-kenya-workers/

https://analyticsindiamag.com/ 13-not-to-miss-research-papers-on-llms/

The above is the detailed content of Is it possible that Bing is integrating GPT-4? Is this true or is it a baseless theory?. For more information, please follow other related articles on the PHP Chinese website!