search
HomeTechnology peripheralsAIDeep learning giant DeepMind has made breakthrough progress on the ImageNet dataset, bringing a new milestone to robotics research

Why does robotics lag so far behind natural language processing (NLP), vision and other artificial intelligence fields? Among other difficulties, data shortage is the main reason. In order to solve this problem, Google DeepMind and other institutions launched the open X-Embodiment data set and successfully trained a more powerful RT-X model

Deep learning giant DeepMind has made breakthrough progress on the ImageNet dataset, bringing a new milestone to robotics research
In 2023, when large models continue to make breakthroughs, research on embodied intelligent robots that use large models as brains to assist in operation is also rapidly advancing.

More than 2 months ago, Google DeepMind launched the first vision-language-action (VLA) model to control robots-RT-2. This model allows the robot to not only interpret complex human instructions, but also to understand the object in front of it (even if the object has never been seen before) and take actions according to the instructions. For example, you ask the robot to pick up the "extinct animal" on the table. It will grab the dinosaur doll in front of it.

Deep learning giant DeepMind has made breakthrough progress on the ImageNet dataset, bringing a new milestone to robotics research

At the time, a Google executive said the RT-2 was a major leap forward in the way robots were built and programmed. "Due to this change, we had to rethink our entire research plan."

What's even more surprising is that only more than two months have passed. DeepMind's robot model has improved again, and it has improved twice as fast.

How is this achieved?

We know that robots are usually very specialized in doing a certain thing, but have poor general capabilities. Typically, you have to train a model for each task, robot, and environment. Changing a variable often requires starting from scratch. But what if we could combine knowledge from various robotics disciplines to create a way to train universal robots?

This is what DeepMind has been doing for some time. They pooled data from 22 different robot types to create the Open X-Embodiment dataset, and then trained a more capable RT-X ( RT-1-X and RT-2-X respectively).

They tested the RT-1-X model at five different research labs, and the results showed that the new method performed better than methods developed independently for each robot. The average success rate increased by 50% across five different commonly used robots. They also show that RT-2-X trained on the above dataset improves performance on real-world robotic skills by a factor of 2, and that by learning new data, RT-2-X masters many new skills. This work shows that a single model trained on data from multiple robot types performs significantly better on multiple robots than a model trained on data from a single robot type.

Deep learning giant DeepMind has made breakthrough progress on the ImageNet dataset, bringing a new milestone to robotics research

It is worth mentioning that this research was not completed by DeepMind independently, but was the result of their collaboration with 33 academic laboratories. They are committed to developing this technology in an open and responsible manner.
Deep learning giant DeepMind has made breakthrough progress on the ImageNet dataset, bringing a new milestone to robotics research
Currently, the Open X-Embodiment dataset and RT-1-X model checkpoints are available to the broad research community.
Deep learning giant DeepMind has made breakthrough progress on the ImageNet dataset, bringing a new milestone to robotics research
Jim Fan, senior artificial intelligence scientist at Nvidia, said today could be the ImageNet moment for robots.

Deep learning giant DeepMind has made breakthrough progress on the ImageNet dataset, bringing a new milestone to robotics research

Google researcher Karol Hausman also expressed the same sigh: The ImageNet moment of robots has finally arrived.

Deep learning giant DeepMind has made breakthrough progress on the ImageNet dataset, bringing a new milestone to robotics research

Open played a key role. Just as ImageNet advanced computer vision research, Open X-Embodiment also advanced robotics.

Building diverse data sets has always been the key to training universal models. These trained models can control many different types of robots, follow different instructions, and perform complex tasks. Tasks perform basic reasoning and generalize efficiently. However, collecting such a dataset would be too resource-intensive for any single laboratory.

To this end, DeepMind collaborated with academic research laboratories at 33 institutions to build the Open X-Embodiment dataset. They collected data from 22 robot instances spanning more than 1 million clips demonstrating the robots' performance in more than 500 skills and 150,000 tasks. This dataset is the most comprehensive robotics dataset of its kind.

##                                Samples from the Open
Deep learning giant DeepMind has made breakthrough progress on the ImageNet dataset, bringing a new milestone to robotics research
#                                               
#RT-1-X: Success rate increased by 50%
Deep learning giant DeepMind has made breakthrough progress on the ImageNet dataset, bringing a new milestone to robotics research
RT-X is built based on two robotics transformer (RT) models become.

Specifically, they used RT-1 to train RT-1-X, where RT-1 is a 35M parameter network built on the Transformer architecture and is designed for robot control The design is shown in Figure 3.
Additionally, they trained RT-2-X on RT-2, a series of large-scale visual language action models (VLA) that perform at Internet scale It is trained on visual and language data as well as robot control data.

To evaluate RT-1-X, DeepMind compared it to models developed on specific tasks, such as opening doors. The results show that RT-1-X trained using the Open X-Embodiment dataset outperforms the original model by 50% on average.


                                                                                                                                                       The average success rate of RT-1-X is 50% higher than the original method.
Deep learning giant DeepMind has made breakthrough progress on the ImageNet dataset, bringing a new milestone to robotics research
## Series from different cooperation agencies about RT-1-X. -X: Barrier-free unlocking of new skills

In order to study the knowledge transfer capabilities of RT-X, DeepMind conducted other experiments. These experiments involved objects and skills that were not present in the RT-2 dataset, but were present in another robot's dataset. Results showed that the RT-2-X was three times more successful at mastering new skills than its previous best model, the RT-2. This also illustrates that joint training with data from other platforms can give RT-2-X additional skills not present in the original data set, allowing it to perform novel tasks.
##                                                                                                                                                                                                                                   
Deep learning giant DeepMind has made breakthrough progress on the ImageNet dataset, bringing a new milestone to robotics research
A series of results show that RT-2-X achieves skills previously unachievable with RT-2, including better handling of space understanding. For example, if we ask the robot to "move the apple near the cloth", or ask the robot to "move the apple to the cloth", in order to achieve the goal requirements, the robot will take completely different trajectories. Simply change the preposition from "near" to "on" to adjust the actions taken by the robot.
RT-2-X shows that incorporating data from other robots into RT-2-X training can improve the robot's task performance range, but only if the use of high enough Capacity architecture.

Deep learning giant DeepMind has made breakthrough progress on the ImageNet dataset, bringing a new milestone to robotics research

                                                                                                                                                                                                                                                                            to be made.

Deep learning giant DeepMind has made breakthrough progress on the ImageNet dataset, bringing a new milestone to robotics researchResearch inspiration: Robots need to learn from each other, and so do researchers

Robot research It’s in the exciting early stages. This new research from DeepMind shows that by scaling learning with more diverse data and better models, it may be possible to develop more useful assistive robots. Collaborating and sharing resources with labs around the world is critical to advancing robotics research in an open and responsible manner. DeepMind hopes to reduce barriers and accelerate research by opening up data sources and providing secure but limited models. The future of robotics depends on robots learning from each other and, most importantly, allowing researchers to learn from each other.

This work proves that the model can be generalized in different environments, and its performance is consistent whether on robots at Google DeepMind or on robots at different universities around the world. has been significantly improved. Future research could explore how to combine these advances with RoboCat’s self-improvement properties, allowing the model to continuously improve based on its own experience. Another future direction is to further explore how mixing different data sets affects cross-embodied agent generalization, and how this generalization is achieved.

If you want to know more about RT-X, you can refer to this paper published by DeepMind:

Paper link: https://robotics-transformer-x.github.io/paper.pdf

Project link: https://robotics-transformer -x.github.io/
Deep learning giant DeepMind has made breakthrough progress on the ImageNet dataset, bringing a new milestone to robotics research
  • Reference link: https://www.deepmind.com/blog/scaling -up-learning-across-many-different-robot-types

The above is the detailed content of Deep learning giant DeepMind has made breakthrough progress on the ImageNet dataset, bringing a new milestone to robotics research. For more information, please follow other related articles on the PHP Chinese website!

Statement
This article is reproduced at:机器之心. If there is any infringement, please contact admin@php.cn delete
Are You At Risk Of AI Agency Decay? Take The Test To Find OutAre You At Risk Of AI Agency Decay? Take The Test To Find OutApr 21, 2025 am 11:31 AM

This article explores the growing concern of "AI agency decay"—the gradual decline in our ability to think and decide independently. This is especially crucial for business leaders navigating the increasingly automated world while retainin

How to Build an AI Agent from Scratch? - Analytics VidhyaHow to Build an AI Agent from Scratch? - Analytics VidhyaApr 21, 2025 am 11:30 AM

Ever wondered how AI agents like Siri and Alexa work? These intelligent systems are becoming more important in our daily lives. This article introduces the ReAct pattern, a method that enhances AI agents by combining reasoning an

Revisiting The Humanities In The Age Of AIRevisiting The Humanities In The Age Of AIApr 21, 2025 am 11:28 AM

"I think AI tools are changing the learning opportunities for college students. We believe in developing students in core courses, but more and more people also want to get a perspective of computational and statistical thinking," said University of Chicago President Paul Alivisatos in an interview with Deloitte Nitin Mittal at the Davos Forum in January. He believes that people will have to become creators and co-creators of AI, which means that learning and other aspects need to adapt to some major changes. Digital intelligence and critical thinking Professor Alexa Joubin of George Washington University described artificial intelligence as a “heuristic tool” in the humanities and explores how it changes

Understanding LangChain Agent FrameworkUnderstanding LangChain Agent FrameworkApr 21, 2025 am 11:25 AM

LangChain is a powerful toolkit for building sophisticated AI applications. Its agent architecture is particularly noteworthy, allowing developers to create intelligent systems capable of independent reasoning, decision-making, and action. This expl

What are the Radial Basis Functions Neural Networks?What are the Radial Basis Functions Neural Networks?Apr 21, 2025 am 11:13 AM

Radial Basis Function Neural Networks (RBFNNs): A Comprehensive Guide Radial Basis Function Neural Networks (RBFNNs) are a powerful type of neural network architecture that leverages radial basis functions for activation. Their unique structure make

The Meshing Of Minds And Machines Has ArrivedThe Meshing Of Minds And Machines Has ArrivedApr 21, 2025 am 11:11 AM

Brain-computer interfaces (BCIs) directly link the brain to external devices, translating brain impulses into actions without physical movement. This technology utilizes implanted sensors to capture brain signals, converting them into digital comman

Insights on spaCy, Prodigy and Generative AI from Ines MontaniInsights on spaCy, Prodigy and Generative AI from Ines MontaniApr 21, 2025 am 11:01 AM

This "Leading with Data" episode features Ines Montani, co-founder and CEO of Explosion AI, and co-developer of spaCy and Prodigy. Ines offers expert insights into the evolution of these tools, Explosion's unique business model, and the tr

A Guide to Building Agentic RAG Systems with LangGraphA Guide to Building Agentic RAG Systems with LangGraphApr 21, 2025 am 11:00 AM

This article explores Retrieval Augmented Generation (RAG) systems and how AI agents can enhance their capabilities. Traditional RAG systems, while useful for leveraging custom enterprise data, suffer from limitations such as a lack of real-time dat

See all articles

Hot AI Tools

Undresser.AI Undress

Undresser.AI Undress

AI-powered app for creating realistic nude photos

AI Clothes Remover

AI Clothes Remover

Online AI tool for removing clothes from photos.

Undress AI Tool

Undress AI Tool

Undress images for free

Clothoff.io

Clothoff.io

AI clothes remover

Video Face Swap

Video Face Swap

Swap faces in any video effortlessly with our completely free AI face swap tool!

Hot Tools

SecLists

SecLists

SecLists is the ultimate security tester's companion. It is a collection of various types of lists that are frequently used during security assessments, all in one place. SecLists helps make security testing more efficient and productive by conveniently providing all the lists a security tester might need. List types include usernames, passwords, URLs, fuzzing payloads, sensitive data patterns, web shells, and more. The tester can simply pull this repository onto a new test machine and he will have access to every type of list he needs.

WebStorm Mac version

WebStorm Mac version

Useful JavaScript development tools

Atom editor mac version download

Atom editor mac version download

The most popular open source editor

EditPlus Chinese cracked version

EditPlus Chinese cracked version

Small size, syntax highlighting, does not support code prompt function

DVWA

DVWA

Damn Vulnerable Web App (DVWA) is a PHP/MySQL web application that is very vulnerable. Its main goals are to be an aid for security professionals to test their skills and tools in a legal environment, to help web developers better understand the process of securing web applications, and to help teachers/students teach/learn in a classroom environment Web application security. The goal of DVWA is to practice some of the most common web vulnerabilities through a simple and straightforward interface, with varying degrees of difficulty. Please note that this software