search
HomeTechnology peripheralsAITencent's robot dog evolves: mastering autonomous decision-making capabilities through deep learning

Tencent's robot dog evolves: mastering autonomous decision-making capabilities through deep learning

Jun 16, 2023 pm 05:01 PM
Tencentdeep learningMake decisions autonomously.

On June 14, Tencent Robotics Decision-making ability has been greatly improved.

Letting robot dogs move as flexibly and stably as humans and animals is a long-term goal in the field of robotics research. The continuous advancement of deep learning technology allows machines to master relevant abilities through "learning" and learn to deal with complex changes. environment becomes feasible.

Introducing pre-training and reinforcement learning: making the robot dog more agile

By introducing pre-training models and reinforcement learning technology, Tencent Robotics Instead of re-learning, you can reuse the multi-level knowledge of posture, environmental perception, and strategic planning that you have learned, and make inferences about other cases from one instance to flexibly respond to complex environments

Tencents robot dog evolves: mastering autonomous decision-making capabilities through deep learning

Tencents robot dog evolves: mastering autonomous decision-making capabilities through deep learning

This series of learning is divided into three stages:

In the first stage, through the motion capture system commonly used in game technology, the researcher collected the movement posture data of real dogs, including walking, running, jumping, standing and other actions, and used these data to construct an imitation learning task in the simulator. , and then abstract and compress the information in these data into a deep neural network model. These models can not only accurately cover the collected animal movement posture information, but also have high interpretability.

Tencent Robotics These technologies and data play a certain auxiliary role in physical simulation-based agent training and real-world robot strategy deployment.

Tencents robot dog evolves: mastering autonomous decision-making capabilities through deep learning

Tencents robot dog evolves: mastering autonomous decision-making capabilities through deep learning

Tencents robot dog evolves: mastering autonomous decision-making capabilities through deep learning

The neural network model only accepts the proprioceptive information of the robot dog (such as motor status) as input and is trained in an imitation learning manner. In the next step, the model incorporates sensory data from the surrounding environment, such as using other sensors to detect obstacles underfoot.

In the second stage, additional network parameters are used to connect the smart posture of the robot dog mastered in the first stage with external perception, so that the robot dog can respond to the external environment through the smart posture that has been learned. When the robot dog adapts to a variety of complex environments, the knowledge that links smart postures with external perception will also be solidified and stored in the neural network structure.

Tencents robot dog evolves: mastering autonomous decision-making capabilities through deep learning

Tencents robot dog evolves: mastering autonomous decision-making capabilities through deep learning

Tencents robot dog evolves: mastering autonomous decision-making capabilities through deep learning

In the third stage, using the neural network obtained in the above two pre-training stages, the robot dog has the prerequisite and opportunity to focus on solving the top-level policy learning problem, and finally has the ability to solve complex tasks end-to-end. In the third phase, additional networks will be added to collect data related to complex tasks, such as obtaining information about opponents and flags in the game. In addition, by comprehensively analyzing all information, the neural network responsible for strategy learning will learn high-level strategies for the task, such as which direction to run, predicting the opponent's behavior to decide whether to continue chasing, etc.

The knowledge learned at each stage above can be expanded and adjusted without re-learning, so it can be continuously accumulated and learned continuously.

Robot Dog Obstacle Chase Competition: Has autonomous decision-making and control capabilities

In order to test these new skills mastered by Max, the researchers were inspired by the obstacle chasing game "World Chase Tag" and designed a two-dog obstacle chasing game. World Chase Tag is a competitive obstacle chasing organization founded in the United Kingdom in 2014. It is standardized from folk children's chasing games. Generally speaking, each round of the obstacle chasing competition involves two athletes competing against each other. One is the pursuer (called the attacker) and the other is the dodger (called the defender). When an athlete competes throughout the entire The team will receive one point when they successfully evade their opponent (i.e. no contact occurs) during the pursuit round (i.e. 20 seconds). The team that scores the most points in the predetermined number of chasing rounds wins the game.

The venue size of the robot dog obstacle chasing competition is 4.5 meters x 4.5 meters, with some obstacles scattered on it. At the beginning of the game, two MAX robot dogs will be placed at random locations in the field, and one robot dog will be randomly assigned the role of a pursuer and the other a dodger. At the same time, a flag will be placed at a random location in the field.

The goal of the dodger is to get as close to the flag as possible without getting caught by the pursuer. The pursuer's task is to catch the evader. If the dodger successfully touches the flag before being caught, the roles of the two robot dogs will instantly switch, and the flag will reappear in another random location. The game ends when the dodger is caught by the current pursuer and the robot dog playing the role of the pursuer wins. In all games, the average forward speed of the two robot dogs is limited to 0.5m/s.

Judging from this game, based on the pre-trained model, the robot dog already has certain reasoning and decision-making capabilities through deep reinforcement learning:

For example, when the pursuer realizes that he can no longer catch up with the dodger before it touches the flag, the pursuer will give up the pursuit and instead wander away from the dodger in order to wait for the next important step. The set flag appears.

In addition, when the pursuer is about to catch the dodger at the last moment, it likes to jump up and make a "pounce" action towards the dodger, which is very similar to the behavior of animals when catching prey, or when the dodger is about to touch the flag. will exhibit the same behavior at times. These are all proactive acceleration measures taken by the robot dog to ensure its victory.

According to reports, all control strategies of the robot dog in the game are neural network strategies. They are learned in simulation and through zero-shot transfer (zero adjustment transfer), allowing the neural network to simulate human reasoning methods to identify never See new things and deploy this knowledge to real robot dogs. For example, as shown in the figure below, the knowledge of avoiding obstacles that the robot dog learned in the pre-training model is used in the game, even though the scenes with obstacles are not trained in the virtual world of Chase Tag Game (only in the virtual world After training in game scenes on flat ground), the robot dog can also successfully complete the task.

Tencent Robotics Learning technology is introduced into the field of robots to improve the control capabilities of robots and make them more flexible. This also lays a solid foundation for robots to enter real life and serve human beings.

The above is the detailed content of Tencent's robot dog evolves: mastering autonomous decision-making capabilities through deep learning. For more information, please follow other related articles on the PHP Chinese website!

Statement
This article is reproduced at:搜狐. If there is any infringement, please contact admin@php.cn delete
Tesla's Robovan Was The Hidden Gem In 2024's Robotaxi TeaserTesla's Robovan Was The Hidden Gem In 2024's Robotaxi TeaserApr 22, 2025 am 11:48 AM

Since 2008, I've championed the shared-ride van—initially dubbed the "robotjitney," later the "vansit"—as the future of urban transportation. I foresee these vehicles as the 21st century's next-generation transit solution, surpas

Sam's Club Bets On AI To Eliminate Receipt Checks And Enhance RetailSam's Club Bets On AI To Eliminate Receipt Checks And Enhance RetailApr 22, 2025 am 11:29 AM

Revolutionizing the Checkout Experience Sam's Club's innovative "Just Go" system builds on its existing AI-powered "Scan & Go" technology, allowing members to scan purchases via the Sam's Club app during their shopping trip.

Nvidia's AI Omniverse Expands At GTC 2025Nvidia's AI Omniverse Expands At GTC 2025Apr 22, 2025 am 11:28 AM

Nvidia's Enhanced Predictability and New Product Lineup at GTC 2025 Nvidia, a key player in AI infrastructure, is focusing on increased predictability for its clients. This involves consistent product delivery, meeting performance expectations, and

Exploring the Capabilities of Google's Gemma 2 ModelsExploring the Capabilities of Google's Gemma 2 ModelsApr 22, 2025 am 11:26 AM

Google's Gemma 2: A Powerful, Efficient Language Model Google's Gemma family of language models, celebrated for efficiency and performance, has expanded with the arrival of Gemma 2. This latest release comprises two models: a 27-billion parameter ver

The Next Wave of GenAI: Perspectives with Dr. Kirk Borne - Analytics VidhyaThe Next Wave of GenAI: Perspectives with Dr. Kirk Borne - Analytics VidhyaApr 22, 2025 am 11:21 AM

This Leading with Data episode features Dr. Kirk Borne, a leading data scientist, astrophysicist, and TEDx speaker. A renowned expert in big data, AI, and machine learning, Dr. Borne offers invaluable insights into the current state and future traje

AI For Runners And Athletes: We're Making Excellent ProgressAI For Runners And Athletes: We're Making Excellent ProgressApr 22, 2025 am 11:12 AM

There were some very insightful perspectives in this speech—background information about engineering that showed us why artificial intelligence is so good at supporting people’s physical exercise. I will outline a core idea from each contributor’s perspective to demonstrate three design aspects that are an important part of our exploration of the application of artificial intelligence in sports. Edge devices and raw personal data This idea about artificial intelligence actually contains two components—one related to where we place large language models and the other is related to the differences between our human language and the language that our vital signs “express” when measured in real time. Alexander Amini knows a lot about running and tennis, but he still

Jamie Engstrom On Technology, Talent And Transformation At CaterpillarJamie Engstrom On Technology, Talent And Transformation At CaterpillarApr 22, 2025 am 11:10 AM

Caterpillar's Chief Information Officer and Senior Vice President of IT, Jamie Engstrom, leads a global team of over 2,200 IT professionals across 28 countries. With 26 years at Caterpillar, including four and a half years in her current role, Engst

New Google Photos Update Makes Any Photo Pop With Ultra HDR QualityNew Google Photos Update Makes Any Photo Pop With Ultra HDR QualityApr 22, 2025 am 11:09 AM

Google Photos' New Ultra HDR Tool: A Quick Guide Enhance your photos with Google Photos' new Ultra HDR tool, transforming standard images into vibrant, high-dynamic-range masterpieces. Ideal for social media, this tool boosts the impact of any photo,

See all articles

Hot AI Tools

Undresser.AI Undress

Undresser.AI Undress

AI-powered app for creating realistic nude photos

AI Clothes Remover

AI Clothes Remover

Online AI tool for removing clothes from photos.

Undress AI Tool

Undress AI Tool

Undress images for free

Clothoff.io

Clothoff.io

AI clothes remover

Video Face Swap

Video Face Swap

Swap faces in any video effortlessly with our completely free AI face swap tool!

Hot Tools

Notepad++7.3.1

Notepad++7.3.1

Easy-to-use and free code editor

Dreamweaver Mac version

Dreamweaver Mac version

Visual web development tools

ZendStudio 13.5.1 Mac

ZendStudio 13.5.1 Mac

Powerful PHP integrated development environment

SAP NetWeaver Server Adapter for Eclipse

SAP NetWeaver Server Adapter for Eclipse

Integrate Eclipse with SAP NetWeaver application server.

DVWA

DVWA

Damn Vulnerable Web App (DVWA) is a PHP/MySQL web application that is very vulnerable. Its main goals are to be an aid for security professionals to test their skills and tools in a legal environment, to help web developers better understand the process of securing web applications, and to help teachers/students teach/learn in a classroom environment Web application security. The goal of DVWA is to practice some of the most common web vulnerabilities through a simple and straightforward interface, with varying degrees of difficulty. Please note that this software