Breaking news: OpenAI is about to open source a new model! Does the prosperity of the open source community depend entirely on the 'charity' of big companies?

Just now, according to the latest news from The Information, OpenAI is about to release a new open source large language model.

Although it is unclear whether OpenAI intends to use the upcoming open source model to seize the market share of Vicuna or other open source models.

But it is almost certain that the capabilities of the new model will most likely not be able to compete with GPT-4 or even GPT-3.5.

After all, the $27 billion valuation also determines that OpenAI’s most advanced models will be used for commercial purposes, although the first two versions of GPT are open source.

A spokesperson for OpenAI did not respond to a request for comment.

Alpaca family open source explosion

Ten days ago, an internal document from Google leaked. In this article titled "We have no moat, and neither does OpenAI", the author laments the heavy blow that open source has dealt to Google and OpenAI.

Indeed, neither Google nor OpenAI seems to be the winner in this arms race, because the open source community is eating up the "benefits" that belong to them .

ChatGPT set off a global LLM revolution. However, OpenAI is not Open, and many companies and developers can only watch and worry.

At this time, Meta stepped forward and released LLaMA, bringing benefits to developers around the world.

Originally, Meta promised that LLaMA would be open source for non-commercial research use cases, but who would have thought that just one week after its release, the weight of LLaMA was suddenly leaked on 4chan. It sparked thousands of downloads instantly.

This "epic leak" directly changed the field of open source LLM. In just a few weeks, various ChatGPT replacements have exploded at lightning speed.

Alpaca, Vicuna, Koala, ChatLLaMA, FreedomGPT, ColossalChat... it can be called an explosion of the "alpaca family".

In fact, long before Yangtuo, the open source model had shattered OpenAI’s ambitions.

At that time, the newly released Dall-E 2 caused quite a stir on the Internet with its stunning Vincentian graphics effect.

However, when OpenAI was still trying to sell APIs, an open source alternative suddenly emerged - Stable Diffusion.

With the rapid rise of Stable Diffusion, Dall-E 2 was quickly forgotten by developers.

Open source big model, is it going to subvert the big companies in Silicon Valley?

UC Berkeley computer professor Ion Stoica is one of the scholars who used Meta's research to develop Vicuna.

To improve Vicuna's capabilities, Stoica and colleagues are working to increase the number of calculations in the model, which will help with tasks involving inference, such as writing code.

Vicuna was developed by a Berkeley team with an annual budget of millions of dollars, about $500,000 of which came from publicly traded companies including Microsoft, Google and Amazon.

Ion Stoica, a professor of computer science at UC Berkeley, said that the performance of the current free AI models is "quite close" to the proprietary models of Google and OpenAI, and there is no doubt that most developers will eventually choose free ones. Model.

On the one hand, open source models allow developers to use their own data to solve specific problems.

On the other hand, the training cost of a model like Vicuna can even be as low as a few hundred dollars, and there is no need to pay expensive usage fees to big manufacturers.

#https://www.php.cn/link/4d8bd3f7351f4fee76ba17594f070ddd

If Stoica is correct, open source AI will definitely subvert the business plans of major companies such as Google, OpenAI, and Microsoft that sell the rights to use proprietary models.

The quality of Vicuna and the Cambrian explosion of open source AI led Google engineer Luke Sernau to warn colleagues that Google was focusing too much on proprietary software in its efforts to catch up with OpenAI.

If free, high-quality alternatives have no usage restrictions, who will pay to use Google products with restrictions? Open source AI is outpacing us, and Google should establish its leadership in the open source community and relinquish some control of our models.

The memo quickly resonated throughout the industry—even if Sernau may have overestimated the capabilities of open source AI and underestimated their costs and risks, most Practitioners agree that Meta has great potential to benefit from this.

For example, Meta uses AI models internally for content recommendation and ad positioning. When developers improve Meta’s models, Meta can incorporate these improvements into its own internal AI.

Meta CEO Xiao Zha has been planning this for a long time.

In a conference call with analysts in April, he said this about the company’s strategy:

It would be better if the industry could standardize on the basic tools we use so we can benefit from others' improvements.

#Google does not adopt a completely proprietary approach to AI software.

Back in 2020, Google released T5, an open source language model that allows developers to build software that can perform translation and summarization tasks. Subsequently, Google released a more advanced Flan-T5.

But according to Stoica and other practitioners, the software released by Meta is able to significantly improve on Google’s model, which makes The possibility of developers choosing the Meta model is greatly increased.

However, Stoica said that Google still has two advantages in open source software.

1. If Google leverages its user data that is not open to the outside world, the model may perform better in certain specialized areas (such as content recommendation).

However, a Google spokesperson said that the company did not train its base model on existing user data.

2. The search company’s expertise in managing large-scale computer infrastructure means it can run models at lower costs, including for cloud customers.

At the same time, OpenAI has already gained a head start in collecting data on how millions of people interact with ChatGPT, which will further help OpenAI improve its AI software, not to mention its cooperation with Microsoft. protocol.

Is the prosperity of open source a "charity" from big manufacturers?

However, this kind of prosperity based on open source is unstable.

Most current open source still relies on giant models released by large companies with deep pockets. If OpenAI and Meta decide to shut down operations, the thriving open source community may become depressed.

#For example, many open source alternatives are now built based on Meta's LLaMA.

Other models use a large public dataset called Pile, compiled by the open source nonprofit EleutherAI.

EleutherAI exists because the openness of OpenAI means that a group of developers can reverse engineer how GPT-3 is made and then create their own models in their free time.

But everything can change.

OpenAI is no longer Open, and Meta is also considering restricting open source to prevent startups from using open source code to do bad things.

Joelle Pineau, executive director of Meta AI, said that opening the code to outsiders is the right thing to do now, but he is not sure that Meta will adopt the same strategy in the next five years.

If this trend of close continues, not only will the open source community be abandoned, but the next generation of AI breakthroughs will also return to the largest and least expensive AI laboratories. in hand.

Clearly, the future of how AI large models are manufactured and used is at a crossroads.

If OpenAI had been stingy, there would be no open source event today

Others are also weighing in. This kind of open source free competition brings greater returns. Or the risk is greater.

At the same time that Meta AI released LLaMA, Hugging Face launched an access control mechanism. Before downloading models on the platform, users must apply for access and obtain approval. This is to limit those who have People with legitimate reasons.

# "I'm not an open source evangelist," said Margaret Mitchell, chief ethics scientist at Hugging Face. "I can see the meaning of not being open source."

One drawback of the widespread use of large models is that it may lead to the proliferation of AI porn products.

Mitchell once worked at Google and founded the AI ​​ethics team, and she is well aware of the risks of model abuse. Therefore, she favors Meta AI releasing models in a controlled manner.

At the same time, OpenAI is also turning off the tap. When GPT-4 was released, it did not announce details such as architecture (including model size), hardware, training calculations, data set construction, training methods, etc. The reason was "in view of the competitive landscape and security impact of large-scale models like GPT-4." .

This restriction reflects the change in OpenAI’s mentality. Co-founder and chief scientist Ilya Sutskever said OpenAI’s past openness was a mistake.

Sandhini Agarwal, policy researcher at OpenAI, said: “Before, if something was open source, maybe a small group of tinkerers would care. But now, the whole environment has changed. Open source is really It can accelerate development and lead to competition."

Going back three years, if OpenAI had adhered to the same principles when it announced the details of GPT-3, there would not have been the emergence of EleutherAI, and there would have been no vigorous open source innovation.

Today, EleutherAI plays a pivotal role in the open source ecosystem. Pile is used to train multiple open source projects, including Stability AI’s StableLM.

But with GPT-4, 5, and 6 being locked, the open source community may once again be left behind several large companies.

They will be stuck in the previous generation model, and if they want to make progress, they can only do it behind closed doors.

The above is the detailed content of Breaking news: OpenAI is about to open source a new model! Does the prosperity of the open source community depend entirely on the 'charity' of big companies?. For more information, please follow other related articles on the PHP Chinese website!