ChatGPT, which has stirred up 10 billion US dollars, is so dependent on TA?

In the past few days, the news that "Microsoft has invested tens of billions of dollars in OpenAI" has been flooding the screen.

As a company with no business model, how can OpenAI be valued at tens of billions of dollars? Are investors so easy to fool?

In fact, if you pull out the ChatGPT behind OpenAI, everyone can understand it more or less. This language model is "top-notch", and it can be said that no one knows about it recently.

And Microsoft, which is close to the water and comes first, has quietly made its first move in the melee of Internet giants.

Just after announcing that ChatGPT will be integrated into its own search engine Bing, the latter announced that ChatGPT plans to join the Office “family bucket”.

The AIGC revolution has reached the stage of computing power again

2022 is the first year of the AIGC revolution.

The birth of DALL·E 2 made "Wen Sheng Tu" popular for a whole year, followed closely by Stable Diffussion and Midjourney, which inspired countless people's artistic inspiration, and even made Many painters feel the "unemployment crisis".

At the end of the year, ChatGPT set off a storm of AIGC among the people. Although it was the finale, the magic given to it by "reinforcement learning" (RLHF) successfully set off a nationwide carnival.

Although the applications we just mentioned look fancy, they are still supported by well-known large models behind them.

As we all know, for large models, whether it is early training or later inference, they are inseparable from a huge amount of "computing power" .

For example, the "popular fried chicken" ChatGPT and DALL·E 2 are based on GPT-3, as well as domestic self-developed source 1.0, Wudao and Wenxin, etc., not only in terms of parameters The volume has reached hundreds of billions, and the data set size has reached TB level.

If you want to handle the training of these "monsters", you need to invest at least more than 1000 PetaFlop/s-day (PD) of computing resources.

It’s no wonder that before OpenAI refined GPT-3, it asked Microsoft to spend $1 billion to exclusively customize a supercomputer that ranked among the top five in the world at the time.

But the problem is that not every company or university that needs a lot of AI computing power can afford to spend huge sums of money to build its own "artificial intelligence high-performance computing center" .

So, if we change our thinking and make computing power more "universal and beneficial", can we achieve the same or better results?

So, in December 2020, the National Information Center and Inspur Information jointly released the "Intelligent Computing Center Planning and Construction Guide". Among them, a brand new concept was mentioned-intelligent computing center (intelligent computing center for short).

#What is the Intelligent Computing Center?

In order to better solve problems in the field of artificial intelligence, the development of intelligent computing centers needs to be based on the latest AI theory and advanced AI computing architecture, and with AI chips, AI servers, and AI clusters As a carrier of computing power.

First of all, the current mainstream AI accelerated computing mainly uses CPU systems equipped with heterogeneous AI acceleration chips such as GPU, FPGA, and ASIC.

Due to the large number of arithmetic logic units (ALU) in the GPU chip originally designed for graphics computing, it can provide a good acceleration effect for deep learning calculations based on tensor calculations, so it is widely used in academia and industry. world welcome.

With more and more in-depth applications, the GPU chip itself has also carried out targeted innovative designs based on the computing characteristics of AI, such as Zhang Quantity calculation unit, TF32/BF16 numerical accuracy, Transformer Engine, etc.

The more "specific" AI computing acceleration chip is mainly derived from GPU chips.

Tesla Dojo Artificial Intelligence Training Chip

Secondly, as The AI ​​server of the computing unit of the Intelligent Computing Center adopts a heterogeneous architecture of CPU AI acceleration chips to achieve ultra-high computing performance by integrating multiple AI acceleration chips. In order to meet the computing needs of scenarios in various fields and complex AI models, AI servers also have extremely high requirements for interconnection and scalability between computing chips.

Finally, the intelligent computing center also needs to provide full support for the industry’s mainstream, open source, and open software ecosystem.

For example, the deep learning frameworks TensorFlow and PyTorch are used to develop AI algorithms, a series of open source libraries built to adapt to the development of specific scenarios such as CV and NLP, etc.

The overall structure of the intelligent computing center

However, compared with those large foreign companies Different from the high-performance computing center customized by the factory, the intelligent computing center is actually an infrastructure for the public, which better solves the problem of being unable to build or use computing power.

The most intuitive one is that the Intelligent Computing OS (Intelligent Computing Center Operating System) as the "nerve center" is to enable the Intelligent Computing Center to efficiently manage the computing resource pool. and intelligent scheduling to better provide external services such as computing power, data, and algorithms.

Today, it has been two years since the release of the 2020 version of the "Intelligent Computing Center Planning and Construction Guide", and the development of my country's intelligent computing centers has also entered a new stage.

To this end, the National Information Center and Inspur Information jointly carried out research that keeps pace with the times, and launched an updated in January 2023 Guidelines for the Innovation and Development of Intelligent Computing Centers" (hereinafter referred to as the "Guidelines").

So, why build an intelligent computing center?

The intelligent computing center is a key infrastructure that promotes industrial transformation and upgrading, optimizes industrial structure, and enhances urban competitiveness in the era of digital economy. It also provides essential infrastructure for the rapidly growing demand for artificial intelligence computing power. support.

Specifically, it can be divided into four aspects:

​Promote AI industrialization, empower It can enable industrial AI, help intelligent governance, and promote industrial clustering.

• AI Industrialization

An important example of AI industrialization, It's autonomous driving.

In the scenario of autonomous driving, it needs to be based on AI technology so that the vehicle can accurately identify the key information in the driving environment like a human driver and make predictions on the potential trajectories of surrounding moving units. Make predictions.

In terms of training computing power consumption, the cognitive model used for autonomous driving is much larger than the general computer vision perception model.

For example, Tesla’s L2 level FSD autonomous driving fusion perception model uses millions of road collection videos during the training process, and the computing power investment is about 500PD.

Moreover, as the level of autonomous driving increases from L2 to L4, the demand for computing power will further increase.

In this regard, the inclusive computing power provided by the intelligent computing center can greatly reduce the cost of computing power required for autonomous driving.

• Industrial AI

# #In 2022, there is a very popular concept called "AI for Science".

This is a new scientific research paradigm, which refers to scientists using AI technology as a production tool.

One of the most well-known examples of AI for Science this year is AlphaFold 2 developed by DeepMind.

So far, AlphaFold 2 can predict more than 200 million protein structures, covering almost the entire "protein universe". Through AlphaFold 2, the prediction time of a single protein structure has been shortened to minutes, and the accuracy has reached 92.4%.

The development of AlphaFold 2 is supported by huge amounts of computing power. In the training data preparation phase alone, AlphaFold 2 consumed about 200 million core-hours of CPU computing power, and about 300 PD of AI computing power during the training process.

The intelligent computing center, which is very suitable for the computing power needs of AI for Science, will become an important infrastructure that supports high-quality scientific research in universities and research institutes.

• Intelligent governance​

##In addition, the intelligent computing center can also provide services to the urban public Services and intelligent governance provide support for intelligent computing capabilities.

• Industrial clustering
##The intelligent computing center can aggregate local artificial intelligence research and development advantages , talent advantages, product advantages and industrial investment to achieve industrial chain synergy and promote the development of industrial clusters.

##Intelligent Computing Center Industry Chain

Now, the Intelligent Computing Center is more It is a key infrastructure to enhance international competitiveness.

We all know that the per capita GDP indicator can measure a country’s economic development, and the level of per capita computing power can also measure a country’s intelligence level.

According to the "2021-2022 Global Computing Power Index Assessment Report", the national computing power index has a significant positive correlation with the trend of GDP. On average, every time the computing power index increases by 1 point, The digital economy and GDP will grow by 3.5‰ and 1.8‰ respectively

##Computing Power Index and GDP Regression Analysis Trend

According to research and calculation, during the "14th Five-Year Plan" period, if the intelligent computing center achieves 80% application level, the city's investment in the intelligent computing center can drive the growth of the core artificial intelligence industry by about 2.9% -3.4 times, driving the growth of related industries by about 36-42 times. The growth in investment in the construction of intelligent computing centers by cities/regions contributes about 14%-17% to innovation output.

It is not difficult to see that the intelligent computing center has formed a force that cannot be ignored for urban economic development.

The economic and social benefits of the intelligent computing center project

For these reasons, once the concept of intelligent computing center was proposed, it triggered a construction boom. According to statistics, more than 30 cities across the country are currently building or proposing to build intelligent computing centers.

However, problems also arise

It cannot be ignored that during the construction process in full swing, these intelligent computing centers have also exposed many problems and challenges .

First of all, our country’s demand for intelligent computing power is increasing day by day.

Data from the "2022-2023 China Artificial Intelligence Computing Power Development Assessment Report" show that the scale of my country's smart computing power will reach 155.2 EFLOPS (FP16) in 2021. It is expected that by 2026, my country's smart computing power will reach 155.2 EFLOPS (FP16). The computing power scale will reach 1,271.4 EFLOPS.

In the future, 80% of scenarios will be based on AI, and most of the computing resources they occupy will be carried by intelligent computing centers.

The development of intelligent computing power in my country

Secondly, around the algorithm The service model also needs to be improved.

Since 2011, global AI leading companies and research institutions have joined in AI large model research, and model parameters have increased dramatically. In just three or four years, the parameter scale has rapidly broken through from billions to trillions, and many representative large models have emerged, such as BERT released by Google, GPT-3 released by OpenAI, etc.

Changes in the traditional computing paradigm will also inevitably promote the transformation of the service model of intelligent computing centers from mainly providing computing power to providing "algorithm computing power."

#Finally, the current conceptual understanding is not yet clear, the construction standards are not unified, the application scenarios are not yet rich, and the operation model is not yet clear. Issues such as maturity also directly affect the development of intelligent computing centers.

How to deal with it?

In this regard, the solution given by the "Guide" is: 1. Universal universal benefit; 2. Open and compatible.

# Let’s first talk about what “universal universal benefit” means. ​

Universal universal benefit refers to giving full play to the social value of public infrastructure, which must not only meet users’ needs for general computing power , but also to meet the needs of different users for diversified computing power in different scenarios.

Specifically, the intelligent computing center should develop in the direction of standardization, low cost, and low threshold, so that intelligent computing can become a basic public service in society like water and electricity to meet the needs of different users. Scenarios require diversified computing power.

#Secondly, it must be "open and compatible." ​

Specifically, it is to open source and cultivate the ecosystem. It mainly focuses on open hardware and open source software, integrates multiple computing power, and realizes the aggregation, scheduling, and release of computing power, so that the intelligent computing center can "use it and use it well." At the same time, it is necessary to strengthen the research and development support and large-scale application promotion of key software and hardware products of the intelligent computing center.

Intelligent Computing Center Construction Structure

In order to realize the universal benefit and efficient utilization of computing power services, that is, how the intelligent computing center can be "easy to use and put to good use", the "Guide" points out that the construction of intelligent computing centers still needs to build "Computing power infrastructure, algorithm infrastructure, service intelligence, and green facilities" 's "four modernizations" technical route.

Computing infrastructure construction means that the intelligent computing center must be able to provide external services with high cost-effectiveness, inclusiveness, The ability of secure computing resources makes AI computing power a basic public resource in the city, like water and electricity, for governments, enterprises, and the public to use independently.

Algorithm infrastructure means that the intelligent computing center provides preset industry algorithms and builds pre-training databases. model, promote the continuous upgrading of algorithm models, provide professional data and algorithm services, allow more users to enjoy universal intelligent computing services, and achieve the effect of "coming with data and leaving with results":

Service smartware means that the development of intelligent computing centers will be upgraded from traditional hardware and software to "smartware" . "Smartware" refers to the middleware products and services provided by the Intelligent Computing Center to promote the application of artificial intelligence. The construction of "smartware" provides users with feature-rich and easy-to-use artificial intelligence computing power scheduling, algorithm supply and personalized development services through visual operation interfaces and low-code development or even no-code development models.

Finally, we must achieve green facilities. The current main development trend is the use of liquid cooling technology.

Looking to the future

Now, emerging scenarios represented by AIGC, Metaverse, and AI for Science are entering our daily lives, giving Our future holds endless possibilities.

Correspondingly, a wave of intelligent computing center construction and layout is rapidly rising across the country.

The ubiquity of intelligent computing power has greatly lowered the threshold for innovation for most enterprises, especially small and medium-sized enterprises.

I believe that in the near future, we will soon witness "data input, intelligence output". Intelligent computing will be like water and electricity, truly benefiting everyone.

Report link: ​

http://scdrc. sic.gov.cn/archiver/SmarterCity/UpFile/Files/Default/20230111181640567177.pdf​

The above is the detailed content of ChatGPT, which has stirred up 10 billion US dollars, is so dependent on TA?. For more information, please follow other related articles on the PHP Chinese website!