The mobile game "Eternal Tribulation" is officially launched! Revealing the black technology behind voice AI teammates｜Qualcomm Snapdragon Gaming Technology Awards 2024 speech transcript

On July 25th, the much-anticipated mobile game "Eternal Tribulation" officially launched its public beta. During the pre-download stage, the mobile game ranked TOP1 on the App Store free list! The newly introduced voice AI teammate function in the game has quickly attracted the attention and praise of the majority of players.

1. NetEase's first AI Copilot Teammate

This innovative gameplay was jointly developed by NetEase 24 Studio and NetEase Fuxi AI Lab. As the world's first game Copilot Teammate, this feature relies on the most cutting-edge AI large model Inference technology has successfully created AI teammates that are comparable to the experience of real players.

2. Feature Highlights

AI teammates can not only independently perform tactical actions such as running maps, searching for supplies, participating in battles, running circles, etc., they can also provide assistance and rescue during battles, and have two-way seamless communication with players through natural voice communicate.

On the day the "Eternal Tribulation" mobile game was launched, NetEase Fuxi AI technical director Ruifeng was invited to give a speech at the "2024 Snapdragon Game Technology Awards", sharing publicly for the first time the technical exploration of voice AI teammates in the "Eternal Tribulation" mobile game and application results. This event is an annual event held by Qualcomm on the eve of ChinaJoy. This year, Qualcomm joined hands with industry partners such as NetEase Fuxi to discuss how mobile technology innovation can empower cutting-edge digital entertainment experiences.

Because the day of the speech coincided with the public beta launch of the mobile game "Eternal Tribulation", Ruifeng was unable to arrive at the scene, so he made an on-site connection.

The following is the transcript of his speech:

Hello everyone, I am Ruifeng from NetEase Fuxi. I am very happy to participate in the "Snapdragon Game Technology Award".

(Ruifeng, head of AI technology at NetEase Fuxi, shared the scene online)

NetEase Fuxi was established in September 2017. It is the first artificial intelligence laboratory in China to conduct AI research for the game industry. With the purpose of "creating a world-class laboratory at the intersection of artificial intelligence and games", it uses cutting-edge artificial intelligence technology to create a new generation of gaming experience for players. It also uses the massive data and simulation environment of the game platform to promote the development of artificial intelligence technology.

Since its establishment in 2017, NetEase Fuxi has implemented a large number of AI gameplay in multiple games. For example, we pioneered the AI ​​photo face pinching gameplay and intelligent parenting system in the PC game "Ni Shui Han". In the PC game "Eternal Calamity", we launched a nightmare human-machine gameplay based on reinforcement learning technology, using AI anti-cheating technology to locate and Punishing offending players ensures a fair gaming environment.

Last year, we implemented a large number of innovative AI applications in the phenomenal mobile game "Ni Shui Han", including smart NPCs, text face pinching, Da Song headlines, UGC crew mode, etc., creating the world's first AI game, and also It has been highly recognized and welcomed by players.

This year, we are the first to release the world's first Voice AI Teammate gameplay in the mobile game "Eternal Calamity" which was officially launched today. In this gameplay, players can invite intelligent AI teammates for a complete game experience. Players can communicate directly with the AI ​​through voice interaction.

First of all, this AI teammate can understand commands, is proficient in searching and delivering equipment, and coordinating in team battles; secondly, it can observe the battle situation in real time, report key information and answer questions; in addition, the AI ​​teammate also has strong combat capabilities, strong gaming skills, Cooperate well; finally, the AI ​​teammates can communicate freely, giving players high emotional value.

In order to realize this AI teammate who can listen, speak and act, NetEase Fuxi based on years of technology accumulation, combined with multiple AI technologies such as speech recognition, semantic understanding, text generation, speech synthesis, reinforcement learning, and imitation learning, A large number of technical problems were solved. In order to achieve a voice interaction experience that is almost the same as that of real teammates, we have set several technical indicators for this gameplay.

First of all, compared to interaction methods triggered by wake words or manual buttons, Quankaimai’s real-time voice interaction poses more technical challenges to us. The core of which is the ability of noise detection and non-interactive content recognition. In order to maximize In order to reduce the rate of false hits, we use NetEase Fuxi's intelligent crowdsourcing platform and adopt an automatic data closed-loop method during the development period to continuously optimize the model effect by continuously collecting real experience data from various models and project members.

Secondly, in order to enable AI to accurately identify player instructions and make the most reasonable behavioral decisions, we not only introduced natural language understanding technology to improve instruction recognition capabilities, but also adopted instruction-based reinforcement learning technology to allow the model to act according to instructions. and real-time battlefield status, making the most reasonable behavioral decisions, achieving a high-intensity gaming ability while following player instructions as much as possible.

最後，為了達到極低的互動體驗速度，我們做了大量的工程最佳化工作。目前線上的即時響應時長最短可以實現300ms內的語音和行為響應，最長響應時長也能控制1s左右，從而實現了讓交流不延遲，讓配合更及時。

正如前面提到的，這個語音AI隊友玩法對於時延非常敏感，為了給用戶帶來更極致的交互體驗，伏羲跟高通進行了深入合作，將部分AI推理從雲端GPU遷移到第三代驍龍8移動平台的NPU上。

為什麼我們要去做端側推理？首先就是低時延，相較於基於雲端的推理，端側推理可以節省網路傳輸帶來的時間延遲，特別是對於一些網路條件不太好的地區的玩家，這部分的時延降低帶來的收益是非常可觀的，從而為這部分玩家帶來更好的遊戲體驗。其次從公司層級來說，AI推理從雲端遷移到端側，可以充分利用玩家的終端算力，降低服務運作成本。我們知道目前雲端的GPU服務都是比較貴的，為了控製成本，很多AI服務都會對應用做一些使用限製或讓玩家採用付費使用的方式，而端側推理可以讓這部分玩家自由地體驗AI玩法。

目前在《永劫無間》手遊的AI隊友玩法中，網易伏羲跟高通已經遷移了部分AI能力到端側，目前正在做更多深入合作。我們預期未來可以將更多AI能力，包含大語言模型（LLM），語音合成（TTS）等技術適配到手機終端，為玩家帶來更好的遊戲體驗。

The above is the detailed content of The mobile game "Eternal Tribulation" is officially launched! Revealing the black technology behind voice AI teammates｜Qualcomm Snapdragon Gaming Technology Awards 2024 speech transcript. For more information, please follow other related articles on the PHP Chinese website!