Huawei's large model appears in the official issue of Nature! Reviewer: Let people reexamine the future of forecast models

It is 10,000 times faster than the traditional method and only takes 1.4 seconds to complete the 24-hour global weather forecast - it is from Huawei Cloud's Pangu Weather Model.

Today, it was published in Nature. It is said to be the first official Nature paper published in recent years with a Chinese technology company as the sole signature unit (that is, the sole author of Huawei Cloud).

Picture

The reviewers gave it high praise. This model allows humans to re-examine the future of weather forecast models.

The implication is that with it, the original traditional methods are no longer fragrant.

Picture

So, how was it developed? What key challenges were solved? What are the specific results and applications?

Follow this paper and take you through it all.

Solving the problem of insufficient accuracy of existing AI weather forecast models

Since the 1920s, especially in the past three decades, with the rapid development of computing power, traditional numerical weather forecasting has Great success has been achieved in weather forecasting, extreme disaster warning, climate change prediction and other fields.

However, as the growth of computing power slows down and the physical model gradually becomes more complex, the bottleneck of this method becomes increasingly prominent.

So researchers began to explore new weather forecasting paradigms such as using deep learning methods to predict future weather.

The Huawei Cloud R&D team started research in this area 2 years ago.

They found that in the fields where numerical methods are most widely used, such as medium and long-term forecasting, the accuracy of existing AI forecasting methods is still significantly lower than numerical forecasting methods, and suffers from lack of interpretability and inaccurate extreme weather predictions. constraints and other issues.

There are two main reasons why the AI ​​weather forecast modelis insufficient in accuracy:

First, existing AI weather forecast models are based on 2D neural networks and cannot handle uneven 3D weather data well;

Second, AI methods lack mathematical and physical mechanism constraints, so in Iteration errors will continue to accumulate during the iteration process.

Here, Huawei Cloud researchers proposed

3D Earth-Specific Transformer (3DEST) to handle complex and complex Uniform 3D weather data to create a large Pangu weather model.

The main idea is to use a 3D variant of the visual transformer to handle complex uneven meteorological elements, and use a hierarchical temporal aggregation strategy , trained 4 models with different forecast intervals (respectively 1 hour interval, 3 hour interval, 6 hour interval, 24 hour interval) , making the prediction The number of iterations for meteorological conditions at a specific time is minimized, thereby reducing iteration errors and avoiding the consumption of training resources caused by recursive training.

To train each model, the researchers used meteorological data from 1979-2021, sampled on an hourly basis, and trained for 100 epochs.

Each model requires 16 days of training on 192 V100 graphics cards. In fact, even after 100 epochs, these models still have not fully converged.

In other words, with more sufficient computing resources, the accuracy of AI forecasts can be further improved.

In the final reasoning, the Pangu weather model only needs 1.4 seconds to run on a V100 graphics card to complete 24-hour global weather forecast, including geopotential, humidity, wind speed, temperature, sea level pressure, etc. The horizontal spatial resolution reaches 0.25∘×0.25∘, the temporal resolution is 1 hour, covering 13 vertical layers, and can accurately predict fine-grained meteorological characteristics.

As the first AI method whose accuracy exceeds that of traditional numerical forecasting methods, its calculation speed is more than 10,000 times higher than that of traditional numerical forecasting.

Can be directly applied to multiple downstream scenarios

In May this year, the direction of Typhoon "Mawa" received widespread attention.

The Central Meteorological Administration stated that the Huawei Cloud Pangu large model performed well in predicting the path of "Mava" and predicted its diversion path in the eastern waters of Taiwan Island five days in advance.

Picture

At the 19th World Meteorological Congress, the European Meteorological Agency also pointed out that the Huawei Cloud Pangu Weather Model has undeniable accuracy. capabilities, the purely data-driven AI weather forecast model has demonstrated forecast capabilities comparable to the European Center for Medium-Range Weather Forecasts’ operational numerical models.

Florence Habie, Director of the European Center for Medium-Range Weather Forecasts, demonstrated in detail the comparison of the real-time operation inspection of Huawei Cloud Pangu Meteorological Model and the European Center for Medium-Range Weather Forecasts:

In order to explore the ability of AI to capture extreme weather, we studied a case in Finland in February this year, when a cold wave of -29°C was observed. We found that Pangu recognized the seriousness of this event earlier.

Picture

Florence Habiye also emphasized that AI prediction methods consume less resources and provide important opportunities for developing countries because It no longer requires large-scale supercomputing resources and provides a rare opportunity to improve global forecasting capabilities.

As for Huawei Cloud choosing the field of AI weather forecasting as a "breakthrough", on the one hand, weather forecasting, especially the accurate prediction of extreme weather such as heavy rains, typhoons, droughts, and cold waves, is related to the international people's livelihood. On the other hand, meteorology The prediction problem is very complex. AI can mine new atmospheric evolution patterns from massive data, which has huge potential for improvement in accuracy and speed.

It is understood that the WMO 2024-2027 strategic plan to be released by the World Meteorological Organization (WMO) absorbs elements of artificial intelligence, making it an important force in promoting the development of meteorological science and technology.

WMO will also actively promote the demonstration application of AI in the fields of nowcasting and numerical weather forecasting, create an international comparison platform for artificial intelligence product applications, formulate AI meteorological application standards and guidelines, and promote the sharing of artificial intelligence data sets, etc. Related work, explore and leverage the application potential of AI in the field of meteorology, and effectively support the national early warning initiative.

Three Keys to the Future

Finally, how does the Huawei Cloud Pangu Weather Model team view the future of AI weather forecasting?

The answer is three keys:

First,Big data. Huge meteorological data is the cornerstone of AI models. Currently, the large-scale Pangu meteorological model only uses part of the ERA5 reanalysis data. Future AI models will be based on massive and more refined global observation data.

Secondly, Big computing power. The ultra-high resolution of meteorological data poses a huge challenge to the training of AI models. The current input resolution of the Pangu meteorological model is 1440×720×14×5, compared with the commonly used resolution of 224×224×3 for computational vision tasks. About 500 times. As the resolution further increases and the model grows, the computing resources required will also increase rapidly.

Finally, Big model. The complex meteorological laws, ultra-high resolution and huge amount of data all determine that AI weather forecasting requires the use of AI models with extremely high computational costs. At the same time, if you want to continuously iterate the leading AI weather forecast model, a stable cloud environment, work suite and corresponding operation and maintenance are also essential.

Paper address: https://www.nature.com/articles/s41586-023-06185-3

The above is the detailed content of Huawei's large model appears in the official issue of Nature! Reviewer: Let people reexamine the future of forecast models. For more information, please follow other related articles on the PHP Chinese website!