The AI ​​that plays the game well is already treating patients and saving lives.

How can a game AI do the job of a doctor?

And this ability is derived from the experience of playing games.

Here, take a pathological scan of the whole film, and you can find the lesion without going through all the high-power fields of view.

In its opinion, this process is similar to logging in "Minecraft".

It’s all three steps:

• Observe the big environment first
• Lock the small range
• Finally determine the goal.

#And this method is very efficient, 400% of the traditional method.

It is worthy of being the game AI that won the NeurIPS MineRL competition...

So, how did it do it?

How does the game AI help the world?

Before introducing this game AI, let us first understand what is the difficulty in processing pathological slices.

Unlike the imagination, which only requires a glance, the clinical department will first scan and digitize the tissue slices.

After that, what is handed over to the doctor is often a high-resolution image of tens of thousands by tens of thousands of pixels or even higher, which can reach 0.25 microns per pixel.

What the doctor has to do is to find the location of risky lesions with the naked eye in this ultra-large image filled with dense cells and tissues and make a judgment. It can be said to be "looking for a needle in a haystack".

In recent years, no one has tried to use deep learning methods to solve this problem, but the challenges encountered are:

The first ,Although pathological images (WSI) have high resolution,of gigapixel size, they often only have one image-level,label.

Most of the current methods rely on dense sampling of whole slices under high magnification for feature extraction, and information integration of all collected features to achieve full slice diagnosis. The workload can be imagined. Know.

Second, the lesion areas in these images are often very sparse. Most existing methods rely on multi-instance learning frameworks, which require dense sampling of local image patches at high magnifications.

This not only increases the calculation cost, but also leads to weak diagnostic correlation and low data efficiency. A slice often takes dozens of minutes to complete the calculation.

However, this time the "Juewu" team from Tencent discovered a blind spot -

Under the traditional model, although doctors need to see with the naked eye, they often use a microscope to see at low magnification first Scan the film under the microscope and use experience to find any doubtful points before reviewing them with a high-power microscope.

And if this kind of operation is put into the world of AI, wouldn’t it be an optimal path decision-making problem? Isn’t this what reinforcement learning can do?

Reinforcement learning is often used in game AI, and game AI is the strength of Juewu AI. Well, the advantages are closed loop.

Previously, Juewu AI has achieved outstanding results in MOBA, RTS, Minecraft and other types of games by virtue of its optimal path decision-making strategy. It also won the NeurIPS MineRL competition championship at the top AI conference.

At that time, CMU, Microsoft, DeepMind and OpenAI jointly held a competition called MineRL at the top conference NeurIPS, requiring participating teams to train a machine that can dig out diamonds in 15 minutes within 4 days. AI "miners".

Juewu AI from Tencent won the championship with an absolute advantage of 76.97 points, successfully becoming the "fastest mining" AI in the history of the challenge.

The action of finding wood in "Minecraft" is actually similar to the action of finding lesions in pathological slides.

The same process is to look around to collect global information (the pathologist scans the film under a low-power microscope), then locks the perspective (confirmation with a high-power microscope), finds the wood, and performs the collection action (confirming the lesion), and so on.

So, based on this game AI, Tencent researchers launched the latest research result "Juewu RLogist", which means exactly RL (reinforcement learning) ) Pathologist.

So how does Juewu RLogist be implemented?

Decision-making efficiency improved by 400%

Just like the solution ideas of human doctors mentioned above, "Juewu RLogist" uses deep reinforcement learning to find the optimal solution. slice path method.

The benefits of this new method are obvious: it avoids using the traditional exhaustive method to analyze local image tiles, but first decides to find areas of observation value, and obtains representative features across multiple resolution levels. , to speed up the interpretation of the entire film.

By imitating the way humans think, it not only improves the efficiency of watching movies, but also saves costs.

Specifically, the researchers achieved cross-resolution information fusion through conditional feature super-resolution.

Benefiting from conditional modeling, high-resolution features in unobserved areas can be updated based on the pairing of low-resolution and high-resolution features that have been observed.

One of the key steps is to define a reinforcement learning training environment for the field of pathological image analysis. This method uses discretized action space, well-designed image block and completion state reward function to improve the convergence performance of the model and avoid local optimality.

The corresponding training pipeline is as shown in the following algorithm:

From the results, the advantages of Juewu RLogist are very obvious. The researchers selected two classification tasks of whole-film scan images, "lymph node slice metastasis detection" and "lung cancer classification", for benchmark testing.

The results show that compared with typical multi-instance learning algorithms, "Juewu RLogist" can achieve close classification performance when the observation path is significantly shorter, with an average The time is shortened to a quarter, and decision-making efficiency is increased by 400%.

Not only that, this method is also interpretable. After visualizing the decision-making process, the researchers found that Juewu RLogist can play a good role in both medical education and actual scenarios in the future.

Currently, the paper has been accepted by AAAI 2023 and the code has been open source.

It is worth mentioning that the researchers also emphasized that they will continue to optimize along the direction of Juewu RLogist in the future, including enhancing RLogist’s representation learning capabilities by introducing a stronger neural network structure, and using higher-order The RL training method avoids learning wrong observation paths, etc.

Where does "Juewu RLogist" come from?

When it comes to AI "Juewu", many people must be familiar with it.

After all, the AI ​​gameplay in "Honor of Kings" is the "Juewu Challenge".

△The red AI armor has an excellent overall view and can turn the tide of the battle by squatting in the grass.

There are also "Minecraft", 3D-FPS games, etc. , it can be said that he is an old player of the "Juewu" game.

The team behind it, Tencent AI Lab, is also a veteran player in letting AI learn to play games. Since 2016, it has developed AI "Exquisite Art", AI "Excellent Enlightenment", and formed an "Enlightenment" platform.

AI "Jueyi" is a chess and card game player.

Its development began in 2016, starting from Go.

In 2017, "Jueyi" won the championship at the UEC World Computer Go Conference and is now a professional sparring partner for the national team.

In addition, it can also play chess and mahjong. In terms of four-player mahjong, "Jueyi" is the first mahjong in the industry to reach professional standards by international standards, and has won the championship in the IJCAI Mahjong AI Competition.

Following the footsteps of “Jue Yi”, the research and development of “Jue Wu” was launched in 2017.

It emphasizes no longer simple games, but the strategic issues of multi-agent AI facing more complex environments.

In 2018, "Jue Yi" reached the level of amateur players of "Honor of Kings", and in 2019, it reached the level of professional e-sports.

The following "King Jue Wu" also brings "Challenge Jue Wu", "Hero Training Ground" and other gameplay methods to Honor of Kings players, becoming a good helper for players to train and improve their scores.

In addition, "Juewu" played "Minecraft" and won the NeurIPS MineRL competition, successfully becoming the "fastest mining" AI in the history of the challenge.

AI

"Football version" Juewu has also won the online world football championship held by Google.

In the process of making game AI, Tencent AI Lab also developed a platform "Enlightenment" together with Honor of Kings.

That is to say, Tencent’s platform, algorithms, and scenarios will be made available to students and academic circles to allow them to conduct relevant game research. In August 2020, the "Enlightenment" platform organized the first Enlightenment college competition, and this year also released the Honor of Kings 1v1 open research environment.

In fact, the game field has always been regarded as the best experimental field for AI.

From the performance of "Juewu" in recent years, it is not difficult to see that it has accumulated certain capabilities in aspects such as reinforcement learning.

So it is also a general trend in the industry to migrate the best capabilities outwards and put them at the practical application level.

This time, we really can’t say that the game AI is “uneducated and incompetent”.

Paper address: http://arxiv.org/abs/2212.01737

Open source address: https://github.com/tencent-ailab/RLogist

The above is the detailed content of The AI ​​that plays the game well is already treating patients and saving lives.. For more information, please follow other related articles on the PHP Chinese website!