Home >Technology peripherals >AI >Who can make a Chinese version of ChatGPT? How to do it?
In December 2022, ChatGPT was born. OpenAI has changed the paradigm of scientific research and engineering applications with a nuclear bomb-level result. In China, ChatGPT has received widespread attention and profound discussions. In the past month, I visited major universities, research institutes, large factories, start-up companies, and venture capital; from Beijing to Shanghai to Hangzhou to Shenzhen, I talked with all the leading players. The Game of Scale has already begun in China. How can the players at the center of the storm accomplish this given the huge gap between domestic technology and ecology and the world's leading edge? Who can do this?
Qin lost its deer, and the whole world chased it away. ———— "Historical Records·Biographies of the Marquis of Huaiyin"
Every time I come into contact with a startup company, I will ask the same question: "Where is ChatGPT, what do you want to do?" I can probably receive three different answers. The first answer is clear: to build China’s ChatGPT.
1.1 Make China’s ChatGPT
Because it is there, I want to reproduce it and make it domestically change. This is very classic product-oriented Chinese Internet thinking. This idea is also a common business model of the Chinese Internet in the past two decades: first Silicon Valley makes something, and then we copy it.
But the problem here is that, first of all, ChatGPT is not like taxi-hailing software, and the difficulty of reproduction is completely different. From a human perspective, the creation of GPT is the result of continuous research by the world's top scientists and engineers since 2015. OpenAI's chief scientist, Ilya Sutskever[1], deeply believes that AGI can achieve. As a disciple of Turing Award winner Geoffery Hinton, he has been studying deep learning since 2007. He has 370,000 citations, and the articles he has published accurately hit all the key nodes of Deep Learning in the past ten years. Even with such a strong team, it took four years to go from GPT 2 to GPT 3.5. One can imagine the difficulty of its science and engineering.
At the same time, the first generation of ChatGPT is OpenAI based on the basic model of GPT 3.5. It took two weeksAfter finetuning the dialog Throw out the demo. The real strength here is not the product ChatGPT, but the underlying GPT 3.5 basic model. This model is still evolving. The GPT 3.5 series has been updated with three major versions in 2022[2], each major version is significantly stronger than the previous version; similarly, ChatGPT was released two months ago A total of four minor versions have been updated[3], each minor version has obvious improvements over the previous version in a single dimension. All OpenAI models are constantly evolving and getting stronger over time.
This also means that if you only focus on the current ChatGPT product, it is tantamount to trying to find a sword. When ChatGPT appeared, it caused a dimensionality reduction blow to existing voice assistants; if you can't see the evolution of the basic model, even if you spend a year or two working hard to make something similar, the foundation of OpenAI at that time Models are also continuing to become stronger. If they continue to productize and finetune a stronger product with a new stronger basic model, will they be hit by dimensionality reduction again?
The approach of carving a boat and seeking a sword will not work.
1.2 To be China’s OpenAI
The second answer is to be China’s OpenAI. The player who gave this answer jumped out of the classic Chinese Internet product thinking. They not only saw a single product, but also saw the powerful driving force for the continuous evolution of the basic model behind this product, which comes from the density of cutting-edge talentsandadvanced organizational structure.
So, if you want to do this, you must not only see For products, we also need to see the talent team and organizational structure behind it; if ranked by scarcity, 人>>卡>>money.
But the problem here is that different soils encourage innovation to different degrees. When OpenAI was first founded in 2015, its investors believed in AGI, even though no profit was seen at the time. Now that GPT has been developed, domestic investors have also believed in AGI, but their beliefs may be different: Do you believe that AGI can make money, or do you believe that AGI can promote human development? ?
Furthermore, even if OpenAI is born here and will appear tomorrow, can the deal they reached with Microsoft be achieved with domestic cloud computing manufacturers? The training and inference of large models require huge costs and require a cloud computing engine as support. Microsoft can devote all its efforts to let the entire Azure help OpenAI[4]. If this is changed to China, is it possible for Alibaba Cloud to help a startup company?
Organizational structure is very important. Only cutting-edge talents and advanced organizational structures can promote the continuous iteration and evolution of intelligence; but it also needs to adapt to the soil where it is located, and find ways to flourish Methods.
1.3 Exploring the limits of intelligence
The third answer is toExplore the limits of intelligence The limit. This is the best answer I've heard. It goes far beyond the classic Internet product thinking of seeking a sword at every turn. It also sees the importance of organizational structure and density of cutting-edge talents. More importantly, it sees the future, sees model evolution and product iteration, and thinks about how to integrate the most profound and most profound things. Difficult problems are solved with the most innovative approaches.
This involves thinking about the extremes of large models.
Observing the current ChatGPT / GPT-3.5, it is obviously an intermediate state. It still has many significant improvements that can be enhanced, and it will be available soon. Strengthened points include:
The input box of the model can be lengthened and the size of the model can be continued As the model increases, the data of the model can continue to increase, multi-modal data can be integrated, the degree of specialization of the model can continue to increase, and all these dimensions can continue to be pulled upward. The model has not yet reached its limit. Limit is a process. How will the model's capabilities develop during this process?
Log-linear curve: The growth of some capabilities will follow the log-linear curve
[8]#After thinking clearly about the limit process, you can reverse the intermediate process from the limit state. For example, if we want to increase the size of the input box:
In this way, we can get a clear technology roadmap for each intermediate stage from the current technology to the limit of scaling
.2.3 Productization according to the model evolution process
The model is constantly evolving, but productization does not need to wait until the final one Model Completion - Whenever a large version of the model is iterated, it can be commercialized. Take the productization process of OpenAI as an example:
##In 2020, the first generation GPT 3 training was completed and the OpenAI API was opened[13]
3. The point at which artificial intelligence significantly surpasses humans
So far, we have discussed the need to analyze the model from the perspective of model evolution and discuss it with extreme thinking. The evolution of the model. Points that can be immediately enhanced at this stage include the length of the input box, larger models and data, multi-modal data, and the degree of specialization of the model. Now let us take a longer-term view and think about how the model can be pushed further to the limit in a larger time and space. We discuss:From these perspectives, It is not unimaginable that artificial intelligence will surpass humans. This raises the next question: How to control strong artificial intelligence that far exceeds that of humans? #This problem is what the Alignment technology really wants to solve. At the current stage, the model’s capabilities, except that AlphaGo surpasses the strongest humans in Go, other aspects of AI have not surpassed the strongest. Stronger humans (but ChatGPT may have surpassed 95% of humans in liberal arts, and it continues to grow). When the model has not surpassed humans, Alignment's task is to make the model conform to human values and expectations; but after the model continues to evolve to surpass humans, Alignment's task becomes to find ways to control intelligent agents that far exceed humans. 4.1 Alignment as a method to control intelligent agents that far exceed humans An obvious problem is that when AI surpasses humans, can it still Make him/her stronger/more disciplined through human feedback? Is it out of control at this point? Not necessarily. Even if the model is far better than humans, we can still control it. An example here is between athletes and coaches Relationship: The gold medal athlete is already the strongest human being in his direction, but this does not mean that the coach cannot train him. On the contrary, even if the coach is not as good as the athlete, he can still make the athlete stronger and more disciplined through various feedback mechanisms. Similarly, the relationship between humans and strong artificial intelligence may become the relationship between athletes and coaches in the middle and later stages of AI development. At this time, the ability that humans needis not to complete a goal, but toset a good goal, and then measure whether the machine Achieve this goal well enough and provide suggestions for improvement. The research in this direction is still very preliminary. The name of this new discipline is Scalable Oversight[15]. 4.2 Alignment and Organizational Structure #On the road to strong artificial intelligence, not only humans and AI need to be aligned, humans and humans also need a high degree of alignment . From the perspective of organizational structure, alignment involves: In 2017, when I first entered the NLP industry, I spent a lot of effort on controllable generation. At that time, the most so-called text style transfer was to change the sentiment classification of the sentence. Changing good to bad was considered a complete transfer. In 2018, I spent a lot of time studying how to let the model modify the style of sentences from the perspective of sentence structure. I once mistakenly thought that style conversion was almost impossible to accomplish. Now ChatGPT makes style conversion very easy. Tasks that once seemed impossible, things that were once extremely difficult, can now be accomplished very easily with large language models. Throughout 2022, I tracked all version iterations from GPT-3 to GPT-3.5[11], and saw with my own eyes its continuous evolution from weak to strong step by step. This rate of evolution is not slowing down, but is accelerating. What once seemed like science fiction has now become a reality. Who knows what the future holds? The millet is separated, the seedlings of the millet are growing. The pace is slow and timid, and the center is shaking. The millet is separated, the ear of the grain is separated. Walking forward with great strides, the center is like intoxication. ———— "The Book of Songs·Mill"
4. Alignment
5. Conclusion
The above is the detailed content of Who can make a Chinese version of ChatGPT? How to do it?. For more information, please follow other related articles on the PHP Chinese website!