search
HomeTechnology peripheralsAINPU launches an innovative UAV control framework: enabling group chat-style interaction, active perception of the environment, and autonomous control of UAVs

Super generalization ability makes large models a ray of hope for "general artificial intelligence".

However, reading thousands of books is not as good as traveling thousands of miles. In an open environment, large models need to truly "walk" into the physical world in order to truly understand complex tasks and solve practical problems.

Recently, Professor Li Xuelong’s team conducted innovative research on autonomous drone swarms in an open environment. They used domestic large-scale models to successfully realize human-computer and multi-machine dialogue interaction in an open environment, breaking the interaction barriers between humans and machines. This research further expands the application scenarios of local security, allowing large drones to soar in real life

Inspired by human cognitive models, our team summarized the highly autonomous cognitive process as " The three-dimensional interaction of "Thinking Computing-Entity Control-Environment Perception" has been established, and a "group chat-style" control framework for autonomous drones driven by the "Scholar·Puyu" open source large model has been established. We equip each drone with an intelligent brain, allowing the drone group to dynamically collaborate through language communication to achieve intelligent interaction, active perception and autonomous control in open environments and complex tasks. This move improves the autonomy of drone mission execution

In general, the main capabilities of autonomous drone swarms include human-like conversational interaction, active environment perception and autonomous entity control

humanoiddialogueinteraction

NPU launches an innovative UAV control framework: enabling group chat-style interaction, active perception of the environment, and autonomous control of UAVsFigure 1 Drone group chat communication

Explore the interaction between human users and drones, allowing drones to understand complex The user needs in the mission are the prerequisites for realizing autonomous drones.

In response to this, the team proposed a "group chat-style" dialogue interaction method, which converts various information such as sounds, images, and the drone's own status into a natural language dialogue form through a large model, realizing the dialogue between users and Drones, and autonomous and intuitive ways to interact with drones.

In order to improve the stability and safety of complex tasks, the team designed an efficient real-time feedback mechanism. This mechanism enables the drone to report its status through dialogue and seek user confirmation at key nodes in mission execution. At the same time, this mechanism can also greatly improve the efficiency of task execution

Active environment perception

NPU launches an innovative UAV control framework: enabling group chat-style interaction, active perception of the environment, and autonomous control of UAVsFigure 2 Actively discover and approach the target

NPU launches an innovative UAV control framework: enabling group chat-style interaction, active perception of the environment, and autonomous control of UAVsFigure 3 Dynamic environment obstacle avoidance

During flight, the drone actively senses the external environment and adjusts the mission plan in real time, which is a key link in completing complex tasks.

In order to deal with this problem, the team developed a task-guided active perception mechanism and proposed multi-sensor fusion low-altitude search, dynamic obstacle avoidance and visual positioning algorithms

In actual mission execution During the process, we can dynamically adjust the flight path and observation posture of the drone based on the perceived information and mission goals. We can try to perceive the world around us from different angles and positions, and gradually reduce the uncertainty in the environment to achieve efficient information collection and task execution

Autonomous Control

NPU launches an innovative UAV control framework: enabling group chat-style interaction, active perception of the environment, and autonomous control of UAVsFigure 4 Autonomous target grabbing

NPU launches an innovative UAV control framework: enabling group chat-style interaction, active perception of the environment, and autonomous control of UAVsFigure 5 Heterogeneous drone cluster collaborative control

The key research is to explore the form of composite agents , to enhance its ability to handle complex tasks. In the era of large models, this is a key area for new intelligent agents

In order to solve this problem, the R&D team used the drone platform to design end effectors such as grippers, upgrading traditional drones to " "Flying Robot", endows it with grabbing capabilities

At the same time, a heterogeneous UAV cluster collaborative control mechanism is also established, and combined with environmental perception feedback, the flight status of the UAV formation is adjusted in real time so that the cluster can Division of labor and cooperation to perform tasks such as regional search, target positioning and crawling

The team successfully tried to apply the three-dimensional interaction model of biological intelligence "thinking computing-entity control-environment perception" to autonomous agents, forming a large-scale autonomous drone cluster. This kind of cluster uses large-scale language models, drone platforms and a variety of sensors to achieve conversational interaction, active perception and autonomous control. This technology is of great significance for the application in on-site security scenarios such as security inspections, disaster rescue, and air logistics

References: Li Xuelong, Vicinagearth security, Communications of the Computer Society of China, 18(11) ), 44-52, 2022

The above is the detailed content of NPU launches an innovative UAV control framework: enabling group chat-style interaction, active perception of the environment, and autonomous control of UAVs. For more information, please follow other related articles on the PHP Chinese website!

Statement
This article is reproduced at:51CTO.COM. If there is any infringement, please contact admin@php.cn delete
The Hidden Dangers Of AI Internal Deployment: Governance Gaps And Catastrophic RisksThe Hidden Dangers Of AI Internal Deployment: Governance Gaps And Catastrophic RisksApr 28, 2025 am 11:12 AM

The unchecked internal deployment of advanced AI systems poses significant risks, according to a new report from Apollo Research. This lack of oversight, prevalent among major AI firms, allows for potential catastrophic outcomes, ranging from uncont

Building The AI PolygraphBuilding The AI PolygraphApr 28, 2025 am 11:11 AM

Traditional lie detectors are outdated. Relying on the pointer connected by the wristband, a lie detector that prints out the subject's vital signs and physical reactions is not accurate in identifying lies. This is why lie detection results are not usually adopted by the court, although it has led to many innocent people being jailed. In contrast, artificial intelligence is a powerful data engine, and its working principle is to observe all aspects. This means that scientists can apply artificial intelligence to applications seeking truth through a variety of ways. One approach is to analyze the vital sign responses of the person being interrogated like a lie detector, but with a more detailed and precise comparative analysis. Another approach is to use linguistic markup to analyze what people actually say and use logic and reasoning. As the saying goes, one lie breeds another lie, and eventually

Is AI Cleared For Takeoff In The Aerospace Industry?Is AI Cleared For Takeoff In The Aerospace Industry?Apr 28, 2025 am 11:10 AM

The aerospace industry, a pioneer of innovation, is leveraging AI to tackle its most intricate challenges. Modern aviation's increasing complexity necessitates AI's automation and real-time intelligence capabilities for enhanced safety, reduced oper

Watching Beijing's Spring Robot RaceWatching Beijing's Spring Robot RaceApr 28, 2025 am 11:09 AM

The rapid development of robotics has brought us a fascinating case study. The N2 robot from Noetix weighs over 40 pounds and is 3 feet tall and is said to be able to backflip. Unitree's G1 robot weighs about twice the size of the N2 and is about 4 feet tall. There are also many smaller humanoid robots participating in the competition, and there is even a robot that is driven forward by a fan. Data interpretation The half marathon attracted more than 12,000 spectators, but only 21 humanoid robots participated. Although the government pointed out that the participating robots conducted "intensive training" before the competition, not all robots completed the entire competition. Champion - Tiangong Ult developed by Beijing Humanoid Robot Innovation Center

The Mirror Trap: AI Ethics And The Collapse Of Human ImaginationThe Mirror Trap: AI Ethics And The Collapse Of Human ImaginationApr 28, 2025 am 11:08 AM

Artificial intelligence, in its current form, isn't truly intelligent; it's adept at mimicking and refining existing data. We're not creating artificial intelligence, but rather artificial inference—machines that process information, while humans su

New Google Leak Reveals Handy Google Photos Feature UpdateNew Google Leak Reveals Handy Google Photos Feature UpdateApr 28, 2025 am 11:07 AM

A report found that an updated interface was hidden in the code for Google Photos Android version 7.26, and each time you view a photo, a row of newly detected face thumbnails are displayed at the bottom of the screen. The new facial thumbnails are missing name tags, so I suspect you need to click on them individually to see more information about each detected person. For now, this feature provides no information other than those people that Google Photos has found in your images. This feature is not available yet, so we don't know how Google will use it accurately. Google can use thumbnails to speed up finding more photos of selected people, or may be used for other purposes, such as selecting the individual to edit. Let's wait and see. As for now

Guide to Reinforcement Finetuning - Analytics VidhyaGuide to Reinforcement Finetuning - Analytics VidhyaApr 28, 2025 am 09:30 AM

Reinforcement finetuning has shaken up AI development by teaching models to adjust based on human feedback. It blends supervised learning foundations with reward-based updates to make them safer, more accurate, and genuinely help

Let's Dance: Structured Movement To Fine-Tune Our Human Neural NetsLet's Dance: Structured Movement To Fine-Tune Our Human Neural NetsApr 27, 2025 am 11:09 AM

Scientists have extensively studied human and simpler neural networks (like those in C. elegans) to understand their functionality. However, a crucial question arises: how do we adapt our own neural networks to work effectively alongside novel AI s

See all articles

Hot AI Tools

Undresser.AI Undress

Undresser.AI Undress

AI-powered app for creating realistic nude photos

AI Clothes Remover

AI Clothes Remover

Online AI tool for removing clothes from photos.

Undress AI Tool

Undress AI Tool

Undress images for free

Clothoff.io

Clothoff.io

AI clothes remover

Video Face Swap

Video Face Swap

Swap faces in any video effortlessly with our completely free AI face swap tool!

Hot Tools

Atom editor mac version download

Atom editor mac version download

The most popular open source editor

Notepad++7.3.1

Notepad++7.3.1

Easy-to-use and free code editor

Dreamweaver Mac version

Dreamweaver Mac version

Visual web development tools

Safe Exam Browser

Safe Exam Browser

Safe Exam Browser is a secure browser environment for taking online exams securely. This software turns any computer into a secure workstation. It controls access to any utility and prevents students from using unauthorized resources.

SecLists

SecLists

SecLists is the ultimate security tester's companion. It is a collection of various types of lists that are frequently used during security assessments, all in one place. SecLists helps make security testing more efficient and productive by conveniently providing all the lists a security tester might need. List types include usernames, passwords, URLs, fuzzing payloads, sensitive data patterns, web shells, and more. The tester can simply pull this repository onto a new test machine and he will have access to every type of list he needs.