SenseTime and Tsinghua University generalist AI agents unlock 100% of Minecraft! 2 days of training on a single CPU to create a redstone circuit from scratch

In the development process of AI, there is a very interesting but contrary to common sense phenomenon——

"Some tasks that are relatively difficult for humans, such as playing chess, It is relatively easy for AI to achieve. However, in the open world, interacting with the environment, planning and making decisions, which are relatively simple things for humans, AI faces huge challenges."

And this is the Moravec Paradox.

However, now GITM has successfully broken this paradox limitation, made breakthroughs in complex and real-world-like environments, and is able to survive, explore and create like humans!

In the best-selling game "Minecraft" that closely simulates the real world, jointly developed by SenseTime The general AI agent Ghost in the Minecraft (GITM) jointly proposed by researchers from Tsinghua University, Shanghai Artificial Intelligence Laboratory and other institutions can not only play "Minecraft", but also performs better than all previous agents.

##Project homepage: https://github.com/OpenGVLab/GITM

Survive, explore and create like humans

This research is an important step towards artificial general intelligence (AGI).

#Extensive mission coverage

GITM achieved 100% mission coverage on all technical challenges in the main world in "Minecraft" (successfully unlocked 262 items in the complete technology tree), while the sum of all previous agents was only Can cover 30%. (In the past, all agent methods, including OpenAI and DeepMind, only unlocked a total of 78)

High Mission success rate

#On the most watched "Get Diamonds" mission, GITM achieved a success rate of 67.5%, compared to the current best result (OpenAI VPT) An increase of 47.5%.

##Extremely high training efficiency

The training efficiency of GITM has also reached new heights. The number of environment interaction steps is only one ten thousandth of that of existing methods, and training on a single CPU node can be completed in 2 days, which is far lower than the 6480 GPU days required by OpenAI VPT or the 17 GPU days required by DeepMind DreamerV3.

The generalist AI agent "Ghost in the Minecraft" (GITM) plays the game "Minecraft" and survives Start from scratch mode, get all the items in the main world, dig diamonds, and make enchanted books!

GITM can handle various terrains, environments, day and night scenes, and even monsters.

GITM can be further applied In the more complex tasks of "Minecraft", such as shelters, farmland, iron golems needed for survival, redstone circuits needed to create automated equipment, nether portals needed to enter the netherworld, etc.

These tasks demonstrate the powerful capabilities and scalability of GITM, allowing the agent to play in "Minecraft" Survive, develop and explore a more advanced world in the medium to long term.

General Artificial Intelligence Breakthrough Accelerates the AI ​​Industrialization Revolution

The purpose of developing an AI agent GITM that overcomes all technical challenges in "Minecraft" is to build an autonomous A general artificial intelligence direction for learning and mastering entire real-world skills.

GITM breaks the traditional RL-based architecture and adopts a new paradigm of large language model (LLM) as the core of the agent.

This innovation also helps accelerate the research goal of artificial general intelligence (AGI) Realize and develop intelligent agents that can perceive, understand and interact like humans in an open world environment, and bring huge breakthroughs and progress to industries such as robotics and autonomous driving, effectively solving complex environments and various long tails in the real world. issues to promote the industrial implementation of AI technology on a larger scale.

##「Ghost in the Minecraft」（GITM）

Benefit With the strategic layout of "big models and big computing power" to promote the development of AGI (general artificial intelligence) and the full-stack large model R&D system, SenseTime has been able to develop rapidly in the field of multi-modal and multi-task general large models, with "every day" "New SenseNova" large-scale model system is the core, continuously helping innovative technologies to be quickly applied in fields such as smart cars, smart life, smart business, and smart cities, and continuously improving industrial intelligence.

Today, the success of GITM will push the ability of applications such as autonomous driving to handle complex tasks to a higher level and break higher technical ceilings.

The above is the detailed content of SenseTime and Tsinghua University generalist AI agents unlock 100% of Minecraft! 2 days of training on a single CPU to create a redstone circuit from scratch. For more information, please follow other related articles on the PHP Chinese website!