Researchers develop robot that can understand English commands and perform household chores

A team of researchers from Princeton University, Stanford University, and Google used OpenAI’s GPT-3 Davinci model to develop a robot named TidyBot that can understand English instructions and perform household chores. This robot can automatically complete tasks such as sorting laundry, picking up garbage on the floor, and picking up toys according to the user's preferences.

The GPT-3 Davinci model is a deep learning model, part of the GPT model family, that can understand and generate natural language. The model has powerful summarization capabilities and can learn complex object attributes and relationships from large amounts of text data. The researchers used this ability to have the robot place objects based on several example objects provided by the user, such as "yellow shirt in the drawer, dark purple shirt in the closet, white socks in the drawer" and then let the model conclude The user's general preference rules and apply them to future interactions.

The researchers wrote in the paper: "Our basic insight is that the summarization capabilities of LLM (Large Language Model) are a good match for the generalization needs of personalized robots. LLM demonstrates the ability to achieve generalization through summarization Amazing ability to exploit complex object properties and relationships learned from massive text datasets."

They also write: "Unlike traditional methods that require expensive data collection and model training, we show that LLM can Achieve generalization in the field of robotics directly out of the box, leveraging the powerful summarization capabilities they learn from massive amounts of text data."

The researchers demonstrated a robot on the paper's website that can do laundry. Separate into light and dark colors, recycle drink cans, throw away trash, pack bags and cutlery, put scattered items back in their place, and put toys in drawers.

The researchers first tested a text-based benchmark dataset in which user preferences were entered and the model was asked to create personalization rules to determine item attribution. The model summarizes the examples into general rules and uses the summary to determine where to place new items. Baseline scenes are defined in four rooms, each with 24 scenes. Each scene contains between two and five places to place items, and there are an equal number of seen and unseen items for the model to classify. The test achieved 91.2 percent accuracy on unseen items, they wrote.

When they applied this method to a real-world robot, TidyBot, they found that it was able to successfully pick up 85 percent of the objects. TidyBot was tested in eight real-life scenarios, each with a set of ten objects, and the robot was run three times in each scenario. According to IT House, in addition to LLM, TidyBot also uses an image classifier called CLIP and an object detector called OWL-ViT.

Danfei Xu, an assistant professor at Georgia Institute of Technology’s School of Interactive Computing, said when talking about Google’s PaLM-E model that LLM gives robots more problem-solving capabilities. "Most previous mission planning systems relied on some form of search or optimization algorithms, which were less flexible and difficult to build. LLM and multimodal LLM enable these systems to benefit from Internet-scale data and easily use to solve new problems," he said.

The above is the detailed content of Researchers develop robot that can understand English commands and perform household chores. For more information, please follow other related articles on the PHP Chinese website!