AI can "play football", but AI robots can't yet

In November this year, the World Cup will start again.

At that time, no matter whether we play or not, whether we play well or not, many people will start talking about football again.

No way, who calls football the largest sport in the world.

However, compared to the World Cup, scientists are more interested in the Annual Robot Cup 3D Simulation League.

Not long ago, a research team from the British artificial intelligence company DeepMind used an accelerated version of the sports course to simulate decades of football matches through computers to train the AI ​​to learn how to skillfully Control digital humanoid football players.

Relevant research was published in the journal Science Robotics.

Paper address: https://www.science.org/doi/10.1126/scirobotics.abo0235

Obviously, this is not the first time that “AI football players” have entered the public eye.

A Brief History of the Evolution of AI Football Players

As early as 2016, not long after AlphaGo defeated Lee Sedol in Go, Deepmind began to ponder Let AI play football.

In June of that year, DeepMind researchers asked AI to control an ant-shaped object to chase the ball, and then dribble the ball until it was sent to the goal to score.

According to David Silver, the leader of the DeepMind team, with the help of Google’s newly developed asynchronous Actor-Critic algorithm, namely A3C, AI not only completed the project, but also continued to use it throughout the entire process. He needs to be taught about mechanics.

This experiment gave “AI Football Playing” a good start.

By 2019, DeepMind has trained many "Players", which are made from different training plans. DeepMind selected 10 two-person football teams from them.

Each of these 10 teams has 25 billion frames of learning experience, and DeepMind collected 1 million games between them.

Then DeepMind sets up the environment to let multiple AIs play football games together, and sets the rules in advance to reward the entire "football team" instead of encouraging an individual "AI player" results to promote the progress of the entire team.

DeepMind has proven in this way that AI can cooperate with each other.

It seemed that everything was going well, but in 2020, DeepMind's AI players had problems.

According to the information provided by Brain JiTi, during a game, one side’s football robots lined up in a row to shoot at the goal, but the robot goalkeeper was not prepared to defend. He fell to the ground and began to swing his legs wildly.

Do you think this is over? Too naive!

Then the forward robot player did a very confusing dance, stomping his feet, waving his hands, and fell to the ground with a thud.

This scene shocked the audience: they have seen people letting loose, but they have never seen people letting loose like this!

Why is this happening?

This also starts with the principle behind it.

At the beginning of "AI Playing Football", researchers adopted the path of reinforcement learning.

Previously, AlphaGo’s learning was based on supervised learning, that is, training through labeled data sets.

But this method has high requirements on the "cleanness" of the data: once there is a problem with the data, the AI ​​will make mistakes.

In contrast, reinforcement learning imitates the human learning model. AI learns in a "trial and error" manner. It is rewarded when it is right and punished when it is wrong, thereby establishing the correct connection. .

It seems to be much smarter than traditional supervised learning, but it still has loopholes.

For example, AI will have a wrong understanding of rewards and punishments, thus generating strange strategies.

OpenAI once designed a rowing game. The original task of the AI ​​was to complete the race.

The researchers set up two rewards, one is to complete the game, and the other is to collect scores in the environment. The result is that the agent found an area and kept spinning around there to "get points". In the end, it naturally failed to complete the game, but its score was higher.

##This time, what new tricks did the AI ​​play

Although There were overturns when the AI ​​played football, but the researchers did not give up.

It was mentioned at the beginning of the article that DeepMind’s research team has made a new breakthrough in training “AI football players”.

According to the paper description, although the DeepMind team simplified the game rules in this study and limited the number of players in the two teams to 2-3, the "AI football players" can complete breakthroughs with the ball, physical confrontation, and precision Shooting and other actions.

So how do researchers train “AI football players”?

Simply put, it combines supervised learning and reinforcement learning.

In the first step, the AI ​​needs to watch video clips of humans playing football and learn to walk naturally, because the AI ​​does not know what to do on the football field at first.

In the second step, the AI ​​practices dribbling and shooting under the reinforcement learning algorithm.

AI can complete these two steps in about 24 hours.

The third step is to train in the form of a competition. The AI ​​robot conducts a 2-on-2 competition. This step takes 2 to 3 weeks, mainly to let the AI ​​learn teamwork. and more difficult motion control such as predicting passes.

The performance of the “AI football player” this time still made the research team feel quite satisfied.

The DeepMind team believes that this research promotes the advancement of artificial systems towards human-level motor intelligence.

However, the DeepMind team is still relatively sober. They know that this breakthrough still has limitations.

For example, the game is 2v2 instead of the 11v11 commonly used in real football matches. This does not mean that AI can participate in more complex football matches.

In addition, even a simple 2v2 match cannot be used directly on robot hardware.

In other words, scientists have not yet been able to develop a robot that can play football.

Reference: https://www.science.org/doi/10.1126/scirobotics.abo0235

https://sports.yahoo.com/deepmind- ai-now-play-football-134345563.html?guccounter=2

https://www.woshipm.com/ai/3619952.htmlhttp://it.people.com.cn/n1/2016 /0622/c1009-28467916.html​

The above is the detailed content of AI can "play football", but AI robots can't yet. For more information, please follow other related articles on the PHP Chinese website!