Turing Award winner LeCun: ChatGPT has huge limitations and the lifespan of the autoregressive model does not exceed 5 years.

The first half of this year can be said to be the most exciting half of the AI ​​industry.

Behind the rapid development of various GPTs and even the prototypes of AGI, there are two camps of people holding different views.

One group believes that generative AI headed by ChatGPT is very powerful and can drive a wave of revolutionary trends, and there is no problem in continuing to advance.

The other group thinks that we are developing a bit too fast. Not only is it forbidden, but it must be stopped for a while. And in terms of morality, there are too few matching constraints.

As a big winner in the AI ​​field, Yann LeCun has a different view on this.

Total ban? Not feasible

LeCun said he was not surprised by ChatGPT’s performance and did not favor a moratorium on artificial intelligence research.

"This may produce the opposite result than expected."

He said that artificial intelligence, as an amplifier of human intelligence, may be the new The origins of the Renaissance.

ChatGPT This large language model is "autoregressive". The AI ​​is trained to extract words from a corpus of up to 1.4 trillion words and predict the last word in a given sequence of sentences, which is the next word that must appear.

Claude Shannon's relevant research conducted in the 1950s was based on this principle.

The principle has not changed, it becomes the size of the corpus and the computing power of the model itself.

LeCun said, "Currently, we cannot rely on this type of model to generate long and coherent text. These systems are not controllable. For example, we cannot directly ask ChatGPT to generate a paragraph of target people. It is the text of a 13-year-old child.

Secondly, the text generated by ChatGPT is not 100% reliable as an information source. The function of GPT is more like an auxiliary tool. Just like the existing Just like the driving assistance system, you still have to hold the steering wheel when the automatic driving function is turned on.

Moreover, the lifespan of the autoregressive language models we are familiar with today is very short, five years is considered After a period of five years, no one will use the past models anymore.

And our research focus should be on finding a way to make these models controllable . In other words, the AI ​​we want to study is an AI that can reason and plan according to a given goal, and must be able to ensure that its safety and reliability standards are consistent. This kind of AI can feel emotions."

You must know that a large part of human emotions is related to whether the goal is achieved or not, that is, related to some form of expectation.

With such a controllable model, we can generate long and coherent text.

LeCun’s idea is to design enhanced models in the future that can mix data from different tools, such as calculators or search engines.

Models like ChatGPT are only trained on text, so ChatGPT’s understanding of the real world is incomplete. If you want to develop further on this basis, you need to learn some content related to the sensory perception and world structure of the entire world.

And these more complex contents cannot be achieved simply by reading text, which is one of the biggest challenges in the next few years.

Open source is the end

The desire for power is unique to humans. And AI will not have this desire just because it becomes more and more powerful.

Only the human species knows how to formulate laws to ensure that individual actions do not harm common interests too much.

OpenAI was originally an open research project and is now closed. OpenAI made no mention of its work, a reversal not common in the research field.

The problem is that training a language model is expensive, costing tens of millions of euros, so startups cannot afford it.

This is also the main reason for the merger of Microsoft and OpenAI, which requires the group’s joint computing power to improve its future models. This is why DeepMind and Google Brain finally merged.

##LeCun said that ultimately in terms of the market, developers will move towards a common ecosystem of open platforms. It would be bad if only a few companies controlled such technology.

Historically, both Facebook and the renamed Meta have actively promoted open basic research, such as the open source project LlaMa.

In the early 1990s, Sun Microsystems and Microsoft were fighting for the right to operate servers. Remember, all the Internet technologies that have remained established are open source.

LeCun finally said that at present, the key to preventing such open source AI platforms is legal issues. If the EU wants to promote the structuring of the AI ​​industry in the future, then this kind of open source platform is essential.

The above is the detailed content of Turing Award winner LeCun: ChatGPT has huge limitations and the lifespan of the autoregressive model does not exceed 5 years.. For more information, please follow other related articles on the PHP Chinese website!