The first high-profile AIGC summit in China was a grand event! Thousands of words of dry information chatted about the GPT-4 era, condensing the speeches of 21 experts

Zhixixi reported on March 24 that today, GTIC 2023 China AIGC Innovation Summit was successfully held in Beijing!

As the GPT-4 era begins, this summit, with the theme of "AI Creates a New World in the New Era", is the first high-profile innovation event in China focusing on generative AI (AIGC). The summit brought together more than 20 industry, academia and research leaders to engage in intense rounds of technical confrontations and collisions of opinions, helping every industrial person who is forging ahead in the fog to complete a journey of eliminating falsehoods and seeking truth.

Too many questions have flooded our brains in the past two months: Will GPT-4 really bring about an explosion of cognitive intelligence? What is the difference between our AI and ChatGPT? Will a Chinese version of OpenAI be born? Will MaaS (Model as a Service), which is frequently mentioned by major manufacturers, be a definite trend? Will China's AIGC industry be a "big factory winner takes all" or a "hundred flowers bloom"... Many questions were answered at this summit. Most of the answers are different from our unthinking intuition, and even the opinions among experts are completely opposite. This makes these discussions extremely valuable.

From the confrontation of large-scale model products of large manufacturers to the collision of top AI investment opinions, from AIGC's pioneering product trial to the "violent aesthetics" PK of computing power companies, from the debate of "big models vs small models" to " "Why didn't ChatGPT first appear in China?" The soul question... wave after wave of climaxes, the summit was full of seats, extremely popular, and the atmosphere of communication was lively. More than 1,000 people attended the meeting throughout the day, and the number of live broadcasts on the entire network was as high as 4.2 million.

Gong Lunchang, co-founder and CEO of Zhiyi Technology, said at the summit on behalf of the organizer that the AI ​​industry is ushering in a new market opportunity period, and the China AIGC Innovation Summit hopes to provide academic Build a communication platform for the industry, industry and investment community. This summit mainly includes a main forum of the AIGC Summit Forum, and three thematic forums of large models, China-like ChatGPT, and AIGC application innovation.

At the meeting, representatives from academic institutions such as the Chinese Academy of Sciences, industry giants such as Microsoft, Baidu, Kuaishou, NVIDIA, Kunlun Wanwei, and SenseTime, Lanzhou Technology, Moxin Artificial Intelligence, UBTECH, and Zhipu AI , Luchen Technology, Yunzhisheng, Zhujian Intelligence, aiXcoder, Movie Book Technology, Computational Aesthetics (Nolibox) and other entrepreneurial pioneers, as well as guest representatives from investment institutions such as Qiming Venture Capital, Creation Partners Capital, Lanchi Venture Capital, etc., shared It covers the cutting-edge innovation, business prospects, computing power evolution, entrepreneurial opportunities and investment strategies of large models and generative AI.

The following is the essence of the speech at today’s AIGC summit.

1. Speech from the organizer: With the evolution of large models, the AI ​​industry is ushering in a new period of opportunities

Gong Lunchang, co-founder and CEO of Zhiyi Technology, delivered a speech for this summit on behalf of the organizer. The emergence and evolution of key algorithms and models such as pre-trained language models and diffusion models have promoted the rapid development of generative AI, and related products have attracted global attention in a very short period of time.

▲Gong Lunchang, co-founder and CEO of Zhiyi Technology

The past week has been a critical period for the rapid development of AI, with the release of GPT-4 and Wen Xinyiyan , Microsoft integrated GPT into the office family bucket, driving a leap in production tools. The AI ​​industry is ushering in a new period of market opportunities.

Based on this, the China AIGC Innovation Summit hopes to build a communication platform for academia, industry, and investment circles, so that everyone can have in-depth exchanges, stimulate ideas, and promote the implementation of cooperation and innovation.

This year is the 7th year since the establishment of Zhiyi Technology. Zhiyi Technology adheres to the dual-wheel drive of technology and industry, focuses on cutting-edge technologies represented by digitalization and intelligence and their industry applications, and builds two major business systems of industrial media and enterprise services.

Zhiyi Technology has an industrial media matrix represented by Zhixi East, Core Dongxi, and Car Dongxi. It has become an industrial media with unique positioning and high influence and credibility in China. At the same time, in response to the needs of industrial upgrading, it has developed Zhiyi Technology With an enterprise service system with open courses as its core, it cooperates with experts and scholars from outstanding industrial companies and top universities around the world to hold a series of talks and new youth lectures, and cooperates with top domestic and foreign companies to organize customized open courses. So far, more than 600 courses have been completed. Festival and gain good reputation.

2. Large models bring about the rise of cognitive intelligence, and major manufacturers and world-class scientists focus on it

At the morning summit, the founder and CEO of Lanzhou Technology & Vice Director of CCF of the Computer Federation of China Chang Zhouming explained the new paradigm brought by the large model, and Zhang Jiajun, a researcher and doctoral supervisor at the Institute of Automation, Chinese Academy of Sciences, explained how the Zidong Taichu large model understands the mystery of the world.

At the same time, Xu Mingqiang, Chief Technology Officer of Microsoft Omni-Channel Division, led everyone to explore AIGC trends and the application of Microsoft Azure OpenAI services in enterprises; Baidu Group Vice President Yuan Foyu, who had just launched Wen Xinyiyan, came to the scene to discuss How Wen Xinyiyan changed the rules of the game in the cloud computing market.

1. Zhou Ming of Lanzhou Technology: Large models bring about the rise of cognitive intelligence, focusing on nine major aspects

Founder and CEO of Lanzhou Technology, Computer Federation of China Zhou Ming, Vice Chairman of CCF and Chief Scientist of Sinovation Ventures, gave an in-depth explanation of the new paradigm brought by large models.

▲Zhou Ming, founder and CEO of Lanzhou Technology, vice chairman of CCF, China Computer Federation, and chief scientist of Innovation Works

As a technical expert who came out of Microsoft, Zhou Ming said that he was deeply influenced by Microsoft co-founder Bill Gates and believed that large models are bringing about the rise of cognitive intelligence. Large models, especially ChatGPT, represent a practical era for language understanding, multi-round dialogue, and problem solving. At the same time, large models effectively solve the problem of NLP task fragmentation and greatly improve R&D efficiency, marking NLP's entry into the industrial implementation stage.

At present, AI is going through the 2.0 era from specialized models for a single task to general models for a wide range of tasks, and then to the AGI era of general artificial intelligence. The AI ​​2.0 era will first innovate creative content, office methods, search engines, human-computer interaction interfaces, finance and other fields.

Founded in June 2021, Lanzhou Technology has launched a number of large model external product services. It has currently implemented the Mencius large model, AIGC (intelligent creation) platform, machine translation platform, financial NLP platform, etc. This technology and product has been implemented by Flush, China Asset Management and other companies. Combined with ChatGPT-like technology, Lanzhou Technology launched the conversation robot MChat, which can help users complete a variety of tasks in specific scenarios through intelligent dialogue.

Talking about the future direction of the industry, Zhou Ming admitted that the current ChatGPT-like technology still lacks in reasoning, logic, mathematics and arithmetic, and factual errors. In the future, nine major issues related to large models are particularly worthy of attention, involving aspects such as reasoning ability, factual correctness, and Chinese processing capabilities.

2. Microsoft Xu Mingqiang: Cooperating with OpenAI to build a supercomputer

The model parameters are showing an exponential growth trend. Today, doubts about large models will be replaced by new doubts in just 1-2 years. Therefore, Xu Mingqiang, chief technology officer of Microsoft's Omni-Channel Division, firmly believes that the model will still grow rapidly, because only 1/10 of the current high-quality corpus is currently used.

▲Xu Mingqiang, Chief Technology Officer of Microsoft Omni-Channel Division

All of this is inseparable from the support of powerful computing power, which also determines everything The size and parameters of the model that can be trained. Therefore, Microsoft Azure collaborated with OpenAI to build an AI supercomputer designed for large-scale AI training. The computer has 285,000 CPUs and 10,000 GPUs.

Xu Mingqiang likened the large language model to a sponge. Wikipedia, medical or scientific papers are water. Stuffing these papers in will bring about the emergence of its capabilities.

Finally settled on enterprise applications. In the CPU era, the problem that enterprises thought about when applying was how to transform business problems into computing problems, that is, converting applications into computing problems through compilers. Now it has changed to how to transform various industries into computing problems. Business problems are transformed into content processing problems.

Enterprise-level ChatGPT application scenarios include customer service, sales market, content generation, knowledge management, assisted decision-making, etc.

3. Zhang Jiajun, Chinese Academy of Sciences: Revealing the secret of "Purple East Taichu", multi-modal large models are beginning to appear "multi-specialized and multi-functional"

At the meeting, Automation Research of the Chinese Academy of Sciences Zhang Jiajun, researcher and doctoral supervisor of the institute and deputy director of Wuhan Institute of Artificial Intelligence, explained how the "Zidong Taichu" large model understands the technical mysteries of the world.

▲Zhang Jiajun, researcher and doctoral supervisor at the Institute of Automation, Chinese Academy of Sciences, and deputy director of the Wuhan Institute of Artificial Intelligence

Zhang Jiajun talked about deep learning pre-training Large model effects continue to improve. At present, through the "big data and big model" method under self-supervised learning conditions, multi-modal large models are beginning to appear "multi-specialized and multi-functional", making rapid progress in small sample learning, natural language question answering, and cross-modal generation. Large models have driven a wave of innovation, but their energy consumption and cost are extremely high, and their cognitive abilities are still far behind those of humans.

"Zidong Taichu" is the world's first multi-modal large model with 100 billion parameters launched by the team of the Institute of Automation, Chinese Academy of Sciences. Zhang Jiajun said that this model supports multi-task self-supervised learning at the token level, modality level and sample level. Multi-modal weakly associated data is trained on 512 cards for 128 days, while achieving unified modeling of modal understanding and modal generation. "Zidong Taichu" supports cross-modal retrieval and generation examples such as searching for pictures with text, using pictures to generate sounds, and using sounds to generate pictures. For example, if you input a real image, Zidong Taichu can generate a personalized 3D image.

Currently, the team has launched Zidong Taichu Open Service Platform 1.0 and Zidong Taichu Luoshen 1.0 AIGC intelligent generation platform, and integrated resources from industry, academia and research to build an artificial intelligence open source open ecosystem and explore general artificial intelligence Industrialization path.

4, Baidu Yuan Foyu: Wen Xin’s words will change the rules of the cloud computing market. More than 100,000 companies have applied to call

The global AI industry has entered an explosive period. AI is creating a whole new world. In 2021, Baidu CEO Robin Li once said: "When a computer has the ability to understand human natural language, has clear expression capabilities, and has good logical reasoning capabilities, it will be very like a human being. And the memory and computing capabilities of machines Far better than humans. Therefore, AI will definitely revolutionize every industry today."

▲Yuan Foyu, Vice President of Baidu Group

Baidu New Generation Big The key technical support behind the language model and generative AI product Wenxin Yiyan is natural language understanding.

Yuan Foyu, vice president of Baidu Group, emphasized, "Among the world's major manufacturers, Baidu was the first to release it. Microsoft directly called Open AI, and Google opened the test on March 21. Neither Meta nor Amazon have truly Release products of the same type and level."

Wen Xin Yiyan has five core competencies, namely literary creation, business copywriting creation, mathematical logic calculation, Chinese understanding, multi-modal generation, and the ability to continue writing "The Three-Body Problem" and "Dream of Red Mansions"; generate live broadcast copy; point out errors in mathematical problems and give the solution process; strong ability to understand the Chinese language; generate multiple images, audios and videos based on Baidu's self-developed large language model Modal generation capabilities.

Yuan Foyu said that judging from the performance of Wenxin Yiyan, it has the ability to understand human intentions, and the accuracy, logic, and fluency of its answers are gradually approaching human levels.

One week after Wen Xinyiyan’s press conference, more than 100,000 companies have applied for Wenxinyiyan’s API calling service test.

In the era of artificial intelligence, the IT technology stack has a four-layer architecture: application layer, model layer, framework layer, and chip layer. Yuan Foyu said: "Globally, Baidu is the only company with leading products in these four layers."

In addition, the development of large models will bring three new cloud computing, industry model fine-tuning and application development. Big industry opportunities.

3. Will China produce an OpenAI? Investors: I disagree!

Tracing upwards to the application, Zhang Fuzheng, the head of Kuaishou MMU NLP Center and Audio Center, talked about the exploration and application of AIGC technology in Kuaishou; downwards to explore the computing power, Wang Wei, founder and CEO of Moxin Artificial Intelligence, went to the bottom It traced how computing power "evolved" in the AIGC era; in addition, Qiming Venture Partners partner Zhou Zhifeng explained the entrepreneurial opportunities and investment strategies in the new wave of AI from an investment perspective.

1. Kuaishou Zhang Fuzheng: Explore AIGC applications, digital people, audio and video bloom in multiple places

Practice brings true knowledge. At the meeting, Zhang Fuzheng, head of the Kuaishou MMU Natural Language Processing Center & Audio Center, shared Kuaishou’s exploration and application in AIGC by demonstrating the latest cases of AI generating digital people, music, and videos.

##▲ Zhang Fuzheng, head of Kuaishou MMU Natural Language Processing Center & Audio Center

Zhang Fuzheng first talked about his understanding of AIGC from the perspective of workflow, based on " Creators input the ideas they want to express and use AI technology to synthesize content and output it as the starting point of the workflow. Kuaishou has laid out AIGC through platform engines, infrastructure, AI atomic capabilities, AIGC solutions, and landing scenarios.

In the digital human scenario, by simply inputting text, users can generate digital human applications with precise mouth shapes and rich expressions/actions within 5 minutes. It can be used for e-commerce, recruitment, anchors, Training and other scenarios.

In terms of smart music, users can generate lyrics and specific melodies that rhyme and are related to the topic words based on the user input of specified topic words and other information. Applications such as AI singers are also close to the singing level of real people.

In terms of video production, AIGC technology is also of great help to creators. For example, the average consumption of Kuaishou's "one-click movie" video creation exceeds 450 million, and the average daily consumption of copywritten movies exceeds 40 million.

2. Wang Wei of Moxin Artificial Intelligence: Sparse computing becomes the optimal solution for large model implementation, Moxin leads the evolution of AI 2.0 computing power

On the demand side, digital The development of civilization has undergone fundamental changes. Generative AI opens the door to building explosively successful applications, and the demand for computing power in the AI ​​2.0 large model era has been completely subverted. Compared with the AI ​​1.0 era of small models, which focused on the versatility of computing power, large-scale pre-training models have unified structures and focus more on scalability. The growth of computing power and inference speed have become pain points in the development and application of large models.

Wang Wei, founder and CEO of Moxin Artificial Intelligence, said: "It is difficult to meet the exponential growth in computing power by hardware alone. It must integrate software and hardware. In this direction, sparse computing is recognized as the most promising The direction of potential development and implementation." Compared with dense computing, sparse computing can achieve a performance improvement of 1-2 orders of magnitude.

▲Wang Wei, founder and CEO of Moxin Artificial Intelligence

At the beginning of 2021, Moxin’s first high-sparse rate AI computing chip Antoum was tape-out Successfully, Moxin subsequently released the Antoum-based AI computing card series S4, S10, and S30 to support diverse AI application scenarios. Based on Moxin's unique dual sparse algorithm and software-hardware collaboration concept, Moxin's products can achieve both high performance and low power consumption, and the dual sparse algorithm can achieve up to 32 times sparsity in many networks and applications.

Through actual measurement on the 176B open source large model BLOOM, the Ink Core S30 computing card can achieve a generation speed of 25 tokens/second when only using medium and low sparsity rates, and generates 4 S30 The generation speed exceeds 8 A100, greatly accelerating the inference speed.

Wang Wei said: "The rapid development of large models has given AI chip startups the opportunity to challenge giant players, and has a new display stage, using disruptive innovation to bring order-of-magnitude performance breakthroughs."

3, Qiming Venture Capital Zhou Zhifeng: Nearly 60% of generative AI entrepreneurs focus on multi-modal applications, and China’s ecology may create more opportunities

In the past 40 years, with the With the continuous explosive growth of computing power and data, the form of AI technology is changing exponentially. How to predict trends and plan ahead at new nodes?

Zhou Zhifeng, a partner at Qiming Venture Partners, said that this wave of AI driven by ultra-large-scale pre-training models has demonstrated transformative generalization capabilities and emergence phenomena from the underlying technology, solving the problem of the AI ​​1.0 era to a certain extent. There are many problems faced by entrepreneurs, including the fact that AI technology is only a small part of the terminal product, the society lacks reasonable expectations for AI technology, the lack of complete application development infrastructure and environment, and the lack of listed companies and capital market valuation systems. wait. AI has once again become a hot spot for entrepreneurship and investment. Within two years of the release of GPT-3 in 2020, investment in AI companies by global venture capital institutions increased fourfold, with US$1.37 billion in financing in 2022 alone.

▲ Qiming Venture Partners partner Zhou Zhifeng

is different from the founder of Baidu who said that "China will basically never have another OpenAI". Zhou Zhifeng believes that , China and the United States have very different ecological environments for AI base large models, and China has many unique opportunities. In addition to large-scale model directions with high technical barriers, high talent density, and high capital requirements, young entrepreneurs, vertical industry veterans, and AI industry giants all have different entrepreneurial opportunities in multiple dimensions of technology and application. The technology team of Qiming Venture Capital has summarized a "map" of the new wave of AI ecological architecture and key layout areas, from the application of intelligent computing platforms, tool chains, open/closed source large models, self-built models/third-party model construction, etc. Come to venture capital for reference.

According to Qiming Venture Capital’s technology team’s communication statistics with more than 100 companies established after 2020, in the field of generative AI entrepreneurship, 14% of entrepreneurs focus on the underlying technology, and 57% of entrepreneurs focus on For multi-modal applications, 29% of entrepreneurs focus on language applications. Technology-driven startups that can build their own barriers in AI technology and application-oriented startups that can integrate into industrial workflows and provide high commercial value are more likely to stand out.

4. Roundtable Dialogue: ChatGPT detonates the technological revolution, "winner takes all" or "let a hundred flowers bloom"?

How did the roaring ChatGPT set off a new round of technological revolution? In the morning roundtable dialogue, Zhang Guoren, co-founder and editor-in-chief of Zhiyi Technology, discussed with Sun Bin, President & COO of Zhujian Intelligence, Huang Dongyan, voice technology scientist of UBTECH, and Liang Yu, partner of Creation Partners Capital, about technology, industry, and investment. The explosive questions pushed the atmosphere of the scene to a climax.

##▲Round table dialogue session, from left to right: Zhang Guoren, co-founder and editor-in-chief of Zhiyi Technology, Sun Bin, president & COO of Zhujian Intelligence, UBTECH Voice Technology Scientist Huang Dongyan and Creation Partner Capital partner Liang Yu

ChatGPT was born out of nowhere and was praised by Nvidia CEO Jen-Hsun Huang as the “iPhone moment of artificial intelligence”. President Takema Intelligent focuses on the implementation of the natural language understanding track industry. &COO Sun Bin lamented that the popularity of this product was beyond their expectation. The new paradigm change brought about by large models and violent parameters has become the beginning.

Youbixuan is a leading player in empowering artificial intelligence to service robots. "The emergence of GPT has allowed us to see the era of artificial intelligence penetrating into all walks of life and service robots entering thousands of households." said Huang Dongyan, a voice technology scientist at UBTECH.

▲Huang Dongyan, a speech technology scientist at UBTECH

Previously, Yann LeCun, one of the three giants of deep learning, once said that ChatGPT has no special innovation, but is just well combined. Liang Yu, a partner at Creation Partners Capital who focuses on investing in early-stage technology companies, also expressed a similar view. He believes that from a technical perspective, the system integration and engineering of the Transformer architecture have been steadily innovating, and ChatGPT has not achieved a revolutionary breakthrough. . Huang Dongyan agreed with Liang Yu’s point of view, and added that OpenAI discovered the “secret given by God” during the process of integration and trial and error, enabling dialogue to achieve stunning human-like performance.

Subsequently, Zhang Guoren raised a question that attracted much attention: the launch of OpenAI's GPT-4 and the successive releases of Microsoft related products have put a lot of pressure on peers. How long is the leading advantage of Microsoft's OpenAI portfolio expected to be maintained? ?

In this regard, Sun Bin believes that any combination of performance tools depends on how people use and apply them. In the end, people’s output must be used as the criterion. The long-term value of their combination lies in whether it can be used in other industries in the future, so that technology and industry can be perfectly unified. Huang Dongyan said that how far the combination of Microsoft and OpenAI can go depends on its technological innovation capabilities and iterative development speed. Of course, there will also be a large number of AI companies that may emerge with "dark horse" technologies in this process.

##▲Sun Bin, President & COO of Zhujian Intelligence

Generally speaking, Microsoft’s approach actually sets a pattern for the development of this industry. Liang Yu said: "To a certain extent, Microsoft is playing an ecological strategy." Industry applications and user access to the ecosystem can form a "data flywheel", and its rolling speed will become faster and faster.

In the domestic environment, large model training also requires good Chinese language learning corpus to start rolling as a "flywheel".

From Meituan co-founder Wang Huiwen, to Sogou founder Wang Xiaochuan, to Kai-fu Lee announcing the Project AI 2.0 plan, many big names are joining the AIGC entrepreneurial wave. When Zhang Guoren asked whether this wave of entrepreneurship would be a "winner-takes-all" trend like entrepreneurship in the Internet era, or a "let a hundred flowers bloom", several guests agreed with the latter.

Sun Bin said that the situation in the era of big models is different from the era of "big subsidies" in the Internet. AI has formed a relatively fixed pattern of entrepreneurial chains. The new big language model has brought new underlying engines, but the accumulated industry Ecology and models will be reused, and the feature of letting a hundred flowers bloom will continue. He admitted that large language models may still be dominated by large manufacturers in the industry, but key breakthroughs will still depend on scientific and technological innovation companies.

Liang Yu also agrees with this point of view. He believes that while large manufacturers are leading the way, opportunities for start-up companies may lie in the application layer, cutting into vertical fields to implement industrial implementation, reduce costs and increase efficiency. "Startups must learn to avoid the 'footsteps of giant beasts'. All great companies emerge from small gaps."

▲Creation Liang Yu

, a partner at World Partner Capital, talked about the elements that AIGC entrepreneurship should have. Liang Yu said that the current competition, capital environment, and talents for large-model entrepreneurship have undergone qualitative changes compared with the previous two years. The capital density , The density of entrepreneurs and the density of entrepreneurs have soared. Entrepreneurs need to have deep technical skills and industry understanding.

Sun Bin believes that just as the co-founder of OpenAI started his business 8 years ago and used his accumulation of technology, resources, and connections to start a business, current entrepreneurs also need to consider three things: computing power, engineers, and data. Starting a business will get twice the result with half the effort.

Many speakers at the summit pointed to the end of AI development as artificial general intelligence (AGI). How to define general artificial intelligence? What is the ultimate problem it wants to solve?

Huang Dongyan believes that general artificial intelligence and artificial intelligence in vertical fields are actually complementary to each other. Intuitively speaking, large companies will provide a platform for general artificial intelligence, while artificial intelligence in vertical fields is in different industries. Do in-depth development.

Sun Bin said that looking at the present ten years from now, this may be the starting point of general artificial intelligence. Liang Yu also mentioned that this year may be the first year of general artificial intelligence. From today on, general artificial intelligence may slowly penetrate into all aspects of people's production and life like water and electricity.

Zhang Guoren said that perhaps many science fiction movies have already depicted the appearance of general artificial intelligence for us, such as Jarvis in Iron Man and MOSS in The Wandering Earth. Although the shapes are different, they have different levels of intelligence. The admiral will reach or surpass human beings.

##▲Zhang Guoren, co-founder and editor-in-chief of Zhiyi Technology

So, let’s look back at today’s wave of AIGC application innovation from 10 years later. What will be the significance and impact on the technology industry and human society?

Sun Bin believes that if we look at today from 10 years later, this will be a major change in the way computers interact, making people no longer have to learn complex languages. Liang Yucheng also mentioned that this year may be the first year of general artificial intelligence. From today on, general artificial intelligence may slowly penetrate into all aspects of people's production and life like water and electricity.

From an ethical perspective, Huang Dongyan said: "How to stimulate the positive effects of technology based on human nature is very critical. In ten years, maybe everyone will have their own intelligent robot to improve people's efficiency. In the end, we can enjoy life better." Subsequently, safety and ethical issues have become concerns for industry people. Huang Dongyan believes that comprehensive governance is needed from laws and regulations, company self-discipline, public awareness, safety ethics standards, etc.

Zhang Guoren concluded that looking back ten years from now, there should be many things that we are accustomed to now, will become unusual by then. From the current perspective, it is just like people are now accustomed to using electronic payments. , versus the adoption of cash payments a decade ago or earlier.

5. Large model special session: What is the gap between domestic products and GPT-4? Vertical and localized overtaking

In the afternoon's large model forum, Zhang Peng, CEO of Beijing Zhipu Huazhang Technology Co., Ltd., explained the pre-trained large model, the foundation of the generative AI era, Luchen Technology Young Professor You Yang, the founder and president of the Department of Computer Science at the National University of Singapore, discussed the challenges and practices of low-cost training of large AI models.

Shi Jianping, investment partner of Lanchi Ventures, believes that pre-trained large language models are ushering in the era of cognitive intelligence. Xu Tianhao, head of NVIDIA’s consumer Internet industry solution architect, and co-founder of SenseTime & Large Device Business Group President Yang Fan brought insights from the aspects of computing power and algorithm practice.

1. Zhipu AI Zhang Peng: Large models also have Moore’s Law, exploring new paths for GLM

The first speaker of the large model forum is Beijing Zhipu Huazhang Technology Co., Ltd. (referred to as: Zhipu AI) CEO Zhang Peng. As a pioneer across industry and academia, he shared the technical path and implementation progress of pre-training large models.

▲Zhang Peng, CEO of Beijing Zhipu Huazhang Technology Co., Ltd.

Pre-trained large models have become the infrastructure for a new generation of AI applications. Zhang Peng said that there is a Moore's Law in the field of large models: the number of parameters in a single model will increase 10 times every year. When the model parameters reach hundreds of billions, the large model has passed an important threshold of qualitative change, and ChatGPT is a representative.

"We have been working hard." Zhang Peng said that based on the transformation of results from the Department of Computer Science of Tsinghua University, Zhipu AI launched its first multi-modal large model in 2021. Zhang Peng said that training a 100 billion model faces various challenges and requires patience and time.

Zhang Peng said that the evaluation report of the bilingual GLM-130B model jointly developed by Zhipu AI shows that GLM-130B is close to or equal to GPT-3 175B (davinci) in terms of accuracy and fairness indicators. In addition, since last year, Zhipu AI has successively open sourced the AI ​​code tool CodeGeeX and released the ChatGLM-6B open source model that can be run on a single card. At the same time, it has launched free auxiliary writing applications such as "Writing Frog" and personalized dialogue "Xiaodai". product, which greatly lowers the threshold for using large models.

Finally, Zhang Peng emphasized the Model as a Service (MaaS) concept at the meeting, advocating to provide flexible deployment methods from pre-trained large models to APIs to multi-level applications, and from cloud to privatization to all-in-one deployment.

2, Luchen Technology Youyang: Build large model training infrastructure Colossal-AI to reduce the cost of AI large model application implementation

The number of AI model parameters is only a few It has increased tens of thousands of times over the years. In the future, AI may be more intelligent and powerful than the human brain. The challenge facing large model training today is the extremely high training cost.

While big data and large models are simultaneously improving, how to create more effective optimization methods, reduce costs and increase efficiency, achieve scalable and efficient computing, and reduce the application implementation costs of AI large models has become an industry key pain points.

Luchen Technology has built a set of efficient distributed AI large model training infrastructure Colossal-AI. It contains three parts: efficient memory management system, N-dimensional parallel technology and large-scale optimization method.

According to You Yang, the founder of Luchen Technology and the principal of the Department of Computer Science at the National University of Singapore, young professor You Yang revealed that Colossal-AI has become one of the fastest growing software in the global basic software market and has been open sourced on Github: https://github.com/hpcaitech/ColossalAI

▲Young Professor You Yang, founder of Luchen Technology and president of the Department of Computer Science at the National University of Singapore

枞The N-dimensional parallel technology developed by Chen Technology creates a variety of parallel strategies including higher-dimensional tensor parallelism, providing a lower-cost and efficient distributed training solution for large models.

In addition, based on the heterogeneous scheduling system of Colossal-AI, users only need to write a line of code to dynamically manage GPU memory, CPU memory, and hard disk, increasing the model capacity of the hardware by dozens of times.

For example, GPT-3 training with 175 billion parameters previously required 128 GPUs, while Colossal-AI only requires 64 GPUs, significantly reducing the hardware requirements and costs of large models. Under the same equipment, Colossal-AI can help users complete model training faster and reduce costs.

3. Shi Jianping of Lanchi Venture Capital: Cognitive intelligence has become the forefront of AI, and "AI-First application" has become a trend

Investment partner of Lanchi Venture Capital Shi Jianping proposed that large pre-trained language models have opened up the era of cognitive intelligence, and cognitive intelligence has become the next frontier of AI.

▲Shi Jianping, investment partner of Lanchi Ventures

Machines are gaining unprecedented cognitive capabilities such as language generation and understanding, knowledge reasoning, etc. Behind it is AI OS (operating system), basic models and pre-trained models, reinforcement learning technology, etc. are all ready.

How do startups embrace the world of cognitive intelligence? Shi Jianping suggested that not all companies need to build large models. They can also start from the vertical application path, such as using their own data to fine-tune open source models/third-party hosted models; they can also start from the platform path, such as providing downstream vertical application developers Provide platform tools such as training, fine-tuning, management, and services.

He said: "The large pre-trained language model has opened an era of intelligent computing with cognitive intelligence as the core driving force." The foundation of digital civilization is code, and cognitive intelligence will also redefine the construction of software. In this way, "AI-First applications" will become a trend. For example, the new version of Microsoft's search engine Bing has already given excellent examples.

"The current pace of technology iteration is faster than PPT." Shi Jianping joked. In this context, how can companies establish core competitive advantages? In his view, the core will be the use of own data to train fine-tuned artificial intelligence models; at the same time, the integration of the intelligent world and the digital world will bring more imagination.

4, NVIDIA Xu Tianhao: Software and hardware collaboration enables over 50% computing power effectiveness when training GPT3 on large-scale clusters

The improvement of hardware computing power not only relies on the improvement of chip technology, but also relies on accurately capturing the needs and trends of AI model algorithm evolution. Xu Tianhao, head of NVIDIA's consumer Internet industry solution architect, said: "Find the key points of computing acceleration, and Continuously innovate to meet future business needs."

##▲Xu Tianhao, Head of NVIDIA Consumer Internet Industry Solutions Architect

Entering the era of large models, one card It is far from being able to carry the training of a model and requires more individuals to form machine nodes that can cooperate with each other.

NVIDIA introduces NVLink, NVSwitch and IB technologies. The Ampere and Hopper architectures build nodes based on NVSwitch and perform cluster networking through the IB network, so that these powerful individuals can efficiently collaborate to complete the same thing.

The underlying hardware is the base. In order for developers to use the hardware and truly solve the problem, software collaboration is required. Therefore, NVIDIA has been building SDKs and scene applications in the past to solve problems in various industries. Among them, NeMo Framework is designed to solve the problems of large model training and inference deployment.

So, how to evaluate how many resources are needed to train GPT-3? Xu Tianhao explained a formula: time consumed = FLOPS required to make a larger model/effective computing power of the hardware. Based on the effective integration of parallel methods and a series of optimizations, NVIDIA's NeMo Framework can achieve more than 50% effective hardware computing performance during the training of GPT-3.

In addition, in order to accelerate the large-scale deployment of large models in enterprises, NVIDIA NeMo Framework also provides an integrated large model solution based on the integration of FasterTransformer and Triton.

5. Yang Fan of SenseTime Technology: The AI ​​production paradigm has undergone major changes, and artificial intelligence will usher in a more prosperous "era of navigation"

At the meeting, Yang Fan, co-founder of SenseTime and president of the large device business group, explained the path of change and the circle of competence of startups that have emerged from the "small model era".

▲ Yang Fan, co-founder of SenseTime and president of the large device business group

AI is leading a new generation of technological revolution, from AI-assisted scientific research to generative AI has swept through the recently popular conversational robot ChatGPT. Big (calculation) power produces miracles, and quantitative changes lead to qualitative changes. Yang Fan said that the AI ​​production paradigm is undergoing a major change-the era of large models is coming.

Yang Fan said that in the past 10 years of the AI ​​"small model" era, deep learning methods and industrialized small model production tools for solving single problems have gradually matured; in the new big model era, large models are as a service ( MaaS) has become a new theme, giving birth to a new AI paradigm centered on cost reduction of large model infrastructure, computing power and data arms race, and real-time user feedback.

Facing this trend, SenseTime, which has been rooted in AI for nearly 10 years, has developed the SenseCore large device, which enables AI native infrastructure, large model production platform, algorithm model services, and industry applications from four levels: The ultimate large model development experience.

Yang Fan said that SenseTime will combine years of accumulated industry experience to provide high-efficiency, low-cost, and large-scale new generation AI infrastructure products and services, accelerate the production and deployment of hundreds of billions of large models across the stack, and promote data The efficiency of collection, labeling, and management has been greatly improved, and the model iteration cycle has been shortened. At the same time, SenseTime will also provide large model development support services to ensure the implementation of development results.

6. ChatGPT-like special session: The battle between To B and To C, large model VS small model

In the afternoon China ChatGPT-like special forum, Li Xiaohan, co-founder and vice president of Yunzhisheng , Jian Renxian, founder and CEO of Zhujian Intelligence, and Fang Han, CEO of Kunlun Wanwei, respectively explained the development process and industry insights of their large language models.

1. Yunzhisheng Li Xiaohan: AI evolution presents three major laws, and enterprises will move towards two paths, general and vertical.

Co-founder and Vice President of Yunzhisheng Li Xiaohan said that the emergence of ChatGPT for AI startups, the biggest feeling is that AGI may become a reality in the next few years, and they are looking for ways to integrate into the wave of large models.

▲Li Xiaohan, co-founder and vice president of Yunzhisheng

The evolution of AI is showing three laws, from algorithm-centered to model-centered. And the beauty lies in the demise of big projects and intermediate tasks. Intermediate tasks refer to intermediate tasks that had no independent goals in the previous development of AI and will gradually be weakened or disappear.

Before 2022, both giants and AI startups are building large models, and the emergence of ChatGPT has verified the feasibility of this path for the industry. "When the model parameters reach a certain scale, 'emergent' capabilities may appear." Li Xiaohan said, and at the level of cognitive intelligence, after a certain period of development, machines may surpass humans.

At the same time, large models have led to two paths for enterprise development, namely creating general large model services and large models for vertical scenarios. The general large model platform of large manufacturers will provide external services and produce model flywheels at a "cabbage price". Companies facing different scenarios need more vertical large-scale models and pay more attention to the controllability of data and services.

Yunzhisheng will focus on the smart medical industry, launch large models for specific industries, and provide customers with enterprise-customized large models based on the industry. Li Xiaohan said that their vision is to move from the industry version to the enhanced universal version.

2. Jian Renxian of Zhujian Intelligence: "Application of large language model knowledge", the future operating system of general artificial intelligence

Jian Renxian, founder and CEO of Zhujian Intelligence said , large language models will become the operating system of general artificial intelligence, which will bring "two worlds", one is the closed source world led by OpenAI and Microsoft, and the other is the open source world led by Deepmind and Google. He believes that the combination of startups and large companies will enable innovation to reach scale and present a situation where "a hundred flowers bloom".

▲Jian Renxian, founder and CEO of Zhujian Intelligence

He further pointed out that the future software paradigm will be applications driven by natural language technology, driven by "big Language model knowledge application" composition. He believes that the model itself has no actual value, but the segmented applications based on large language models have actual value. Therefore, he leads the development of "Model as an Application" (MaaA) rather than "Model as a Service" (MaaS).

Jian Renxian believes that applications based on ChatGPT will bring great value to white-collar workers and bring value to enterprises, such as customer service automation, virtual assistants, knowledge management and employee training. Four application scenarios: will be disrupted first.

In order to balance the advantages and disadvantages of large language models and small language models, Jian Renxian proposed a formula: "Small natural language processing model knowledge model large model = natural language processing dual engines". He believes that small models can make Large models are more controllable, usable, and interpretable. Currently, Zhujian Intelligence has leveraged its natural language processing expertise to integrate existing scalable products with large-scale language models to provide pre-trained models for various industries.

3. Kunlun Wanwei Fanghan: There are three major gaps in the development of large models between China and foreign countries. Domestic business model innovation begins with enterprise services

Large models start with hundreds of billions of models True general artificial intelligence emerges, and the iPhone moment of AI has arrived. Kunlun Wanwei CEO Fang Han said that ChatGPT may become a milestone in the evolution of carbon-based life to silicon-based life, and it will be the second evolution in human history.

▲Kunlun Wanwei CEO Fang Han

However, the current gap in the development of large models between China and foreign countries includes three aspects. First, the annual cost of 50 million to 100 million US dollars is the ticket for hundreds of billions of large model training. Second, the data quality of Chinese texts is not High, the biggest gap is the engineering technology gap.

Fang Han said that OpenAI’s efforts in the direction of GPT prove that general artificial intelligence is achievable. After GPT-3 was just released and experienced, he believed that GPT-3 represented a milestone for AIGC and would greatly subvert the field of content generation.

At the application level, Fang Han said that the AIGC industry will follow the logic of reducing costs for the B-side and increasing efficiency for the C-side. It can be seen that Microsoft's layout on the B-side is focused on large customers in industries such as finance and energy. The reason is that "the data of these enterprises can be combined with large models such as GPT to generate the next generation production paradigm." On the C side, taking Microsoft's Copilot as an example, it will improve people's productivity.

Therefore, Fang Han predicts that business model innovation in the domestic AIGC industry will first appear in the field of B-side enterprise services, followed by the field of C-side UGC tools.

7. AIGC application innovation special session: AI transformation code generation, video creation, art design...

In the afternoon AIGC application innovation special forum, aiXcoder CTO Hao Yiyang, Director of Movie Group & Beijing District President Fan Shuo and Computational Aesthetics (Nolibox) co-founder Huang Shengyu respectively discussed the collision and integration of AIGC with code generation, metaverse, design and creativity and other industries.

1. aiXcoder Hao Yiyang: GPT-4 brings new changes in code generation, and will promote the expansion of models to hundreds of billions in the future

At the meeting, aiXcoder (Silicon Heart Technology ) CTO Hao Yiyang gave a keynote speech titled "Code Generation in the Era of Large Language Models (LLM)".

He said that GPT-4 has brought new changes in code generation, supporting longer sequences, more instruction number fine-tuning, multi-modal (picture input) and other operations, showing that it is more suitable for general use effect, but also faces many problems, including lack of relevant documents, dependent libraries and requirement documents, slow speed, information security threats, etc.

In fact, there is a big difference between procedural generation models and language models. For example, in terms of interaction methods, ordinary dialogue language models mainly focus on question and answer and continuation, while procedural generation models need to fill in the blanks, complete, and back up. . There are still many things that GPT-4 cannot do in code generation. For example, in terms of real-time performance, GPT-4 is not applicable in some code correction and code completion scenarios that require real-time feedback; context sequences are still limited. It is difficult to take into account all the context of medium and large projects; there is a big gap between the complete information of code projects and the text of web crawling.

As a start-up that entered the AI ​​intelligent programming robot track in 2018, aiXcoder launched the country's first code generation pre-training model product aiXcoder XL in June 2022. The product supports output from full-featured natural language input to full-fledged programming languages.

Looking forward to the aiXcoder roadmap, Hao Yiyang said that aiXcoder will promote the expansion of the model from tens of billions to hundreds of billions, add a large number of mixed data of natural language processing codes, and specially construct instruction data sets for various scenarios in programming. , thereby obtaining a code editing tool with better overall performance.

▲aiXcoder (Silicon Heart Technology) CTO Hao Yiyang

2. Fan Shuo of the Movie Group: Building a digital operating system for content terminals, many The modality will usher in an explosion in the next two years

Generative AI has reached an explosive stage, and the explosion of applications it brings makes people feel the process of AI leaping from perception to cognitive intelligence, and It brings a lot of market space.

ChatGPT has brought the text modality to an explosion period, but the explosion period of pictures, videos, and sounds has not yet arrived in this era. Fan Shuo, director of Movie Book Group and president of Beijing District, said that with the emergence of industry demands and the improvement of production efficiency, pictures, videos and even digital twins will become an irreversible trend in the future, making it more intuitive and convenient for people to obtain information.

▲Fan Shuo, Director of Film Book Group and President of Beijing District

The entire technological transformation is the same as the process of human development, from solving repetitive tasks to thinking about logic and creativity.

Currently, text modalities have brought improvements in people’s productivity. Fan Shuo said: “2023 to 2025 will become an era of multi-modal explosion.” In the future, content built by generative AI will Not only papers and codes, but user-defined and self-interactive production.

In addition, for terminal companies, "models are not directly accessible to many companies." Fan Shuo said, therefore, the model requires more applications to connect and more platforms to support, and then the model needs to be Data structuring, production content standardization and processization can truly be applied to the entire content generation process. Movie Group has developed an AI digital operating system.

The development of the large model industry will continue to adapt to some repetitive work in the field of content generation. In the future, its system will achieve standardization capabilities, while docking with corresponding open engines to build an overall ecology, and continue to radiate text, sounds, pictures, Video and other different modes.

3. Computational aesthetics Huang Shengyu: Grasp the "three elements" of transformation and let AIGC drive design creativity

AI is affecting human art. Huang Shengyu, co-founder of Computational Aesthetics (Nolibox), shared how AIGC drives innovation in design and creative production mechanisms.

▲Huang Shengyu, co-founder of Computational Aesthetics (Nolibox)

Huang Shengyu said that the integration and evolution of design and computing have gone through three stages: The 1.0 era is Machine-assisted design represented by Adobe, the 2.0 era is digital design tools represented by tools such as Canva, and the 3.0 era is direct design based on AIGC. For this purpose, computational aesthetics has launched "Picture Universe", "Painting Universe", etc. tool.

"Intelligent design is to the design industry what autonomous driving is to the transportation industry." Huang Shengyu said, "But design behavior is complex and there is generally no optimal solution. The optimization function is often not unique. Clear."

The realization of controllable design and creative production driven by AIGC faces at least three major challenges: 1. The language incommensurability of humans and machines when it comes to creativity. 2. Obtain and analyze complex design creative requirements. 3. Design-generated creative solutions are difficult to iterate and implement. In this regard, Huang Shengyu proposed "three elements" for the transformation of design cognition into robot cognition: 1. The quantifiable nature of design assets. 2. The generalizability of design experience. 3. Simulation of design behavior.

Founded in 2020, Computational Aesthetics is a technology enterprise incubated by Tsinghua University. It has launched the world's first commercially available visual design data set. Its core products include the AIGC productivity tool "Drawing the Universe" and an intelligent design engine. "Picture Universe" plans to launch the AI ​​design creative platform Yeahpix in the next step.

Conclusion: Generative AI detonates the innovation of content production and interaction paradigm, witnessing the arrival of a new era of AI

GTIC 2023 China AIGC Innovation Summit came to a successful conclusion, but the content production and interaction spurred by large models and generative AI have The interaction paradigm revolution has just begun.

In this AIGC event, we saw that ChatGPT was born out of the development and accumulation of technology in the era of large models, which excited the industry, academia and investment community. At the same time, this phenomenal product has become a milestone in the era of large models, demonstrating the potential for the emergence of intelligence for companies that have been deeply involved in large models for many years.

At the same time, the scale of model parameters is exploding, and the difficulty of training hundreds of billions of models can be imagined. At the AI ​​chip and computing power levels, there are opportunities for large-scale deployment of generative AI products.

Generative AI has brought disruptive application innovations, from text, pictures, codes, videos and even creative production to the metaverse industry. Generative AI has opened up unlimited space for creation and imagination for different industries. More and more ChatGPT-like products are being commercialized, and AI is penetrating into all walks of life, accelerating the era of general artificial intelligence.

Looking to the future, generative AI is leading us into a new world of AI, and the rapidly advancing ChatGPT is setting off a new round of technological revolution.

GTIC 2023 China AIGC Innovation Summit will become an important platform for industry, academia and research circles to conduct in-depth exchanges and stimulate ideas around cutting-edge technologies and industrial implementation. AIGC companies from various subdivisions are invited to become witnesses of the arrival of the new era of AI. By.

The above is the detailed content of The first high-profile AIGC summit in China was a grand event! Thousands of words of dry information chatted about the GPT-4 era, condensing the speeches of 21 experts. For more information, please follow other related articles on the PHP Chinese website!