Actual test of NVIDIA AI game engine: real-time chat with NPC, Chinese is fluent

The intelligent NPC created by Academician Huang in "Cyberpunk 2077" has already become Chinese-speaking?

Qubit’s first-hand experience, witnessed NPCs fluently conversing in Chinese and English, with natural expressions and movements, and matching mouth shapes...

If it weren’t for the fact that there was a screen in front of me , you will really feel like you are on the scene.

At this year’s CES show, Nvidia used its intelligent engine Avatar Cloud Engine (ACE) to make game NPCs “alive”, which caused quite a shock.

△Intelligent NPC displayed at CES

Using ACE, characters in the game can have realistic voice conversations with players while showing vivid expressions and physical movements without having to prepare a script in advance.

At the time of its debut, major domestic and foreign game companies such as Ubisoft, Tencent, NetEase, and MiHoYo announced that they would follow suit.

Now, by adding the translation module, the NPCs in the game can also learn Chinese.

Next, follow the qubit lens to experience the Chinese dialogue version of the cyberpunk world~

Intelligent NPC Kai Biao Chinese

Go as Kai When we entered the ramen shop, we were still greeted by the owner Jin and our friend Nova.

The background of the story is the same as the DEMO shown at CES. Nova went to the ramen shop to tell Jin that her company had just signed a big deal.

We walked up to Nova and asked her to help recommend the best ramen here. Nova told us without hesitation that beef ramen is the best.

Both the content and the expression are very natural, but the Chinese pronunciation is a bit magical, but considering that Nova is a foreigner, it is reasonable.

After we expressed our gratitude, Nova directly ordered a bowl of noodles to Jin and asked us to eat.

After we excused ourselves and said we had to pay, we started chatting. Through the conversation, we learned that Nova is a regular customer of this store.

At the same time, Nova once again told us about her company signing a large order.

After congratulating Nova, we turned our attention to the owner Jin.

The topic still started with ramen. Jin recommended soy sauce ramen and miso ramen to us, and we decided to have a bowl of each.

Jin agreed and brought the prepared ramen in front of us, and we started to take the opportunity to ask about how to make ramen.

Although Jin did not answer our questions in Chinese, it can be seen from what he said that he understood what we said.

However, as the conversation continued, Jin began to jump back and forth between Chinese and English, but there was always no problem with understanding.

Moreover, sometimes even if we do not recognize 100% what we say, it does not affect Jin's understanding of what we want to express.

What Jin said provided new ideas for our subsequent conversation, and we began to ask questions based on Jin's answers.

What’s interesting is that Jin has never forgotten his identity as the owner of a ramen restaurant, and when talking about “new things”, he also promotes the sake in the store.

Following the DEMO at CES, we asked Jin again about his views on artificial intelligence.

The difference is that Jin here does not think that he may be an AI, and then joked:

Even if I am an AI, at least it is an AI that can make ramen, no ?

Of course, this is due to the background setting of this experience game - in ACE, you can preset the character's personality and other information to get different reply styles.

After some experience, the performance of NPCs is impressive enough. They are no longer cold word-reciting machines, but "alive" like real people.

In this, Nvidia’s ACE platform plays an important role.

The real Chinese NPC is not far away?

The engine that drives NPC can be mainly divided into three modules - two of which are from the NVIDIA ACE platform and one from a third party.

Specifically, the ACE microservices of the two modules NVIDIA Riva and NVIDIA Audio2Face are responsible for processing voice and character expressions respectively.

The specific conversation content is generated by the third party Convai using a large model.

Since Convai only supports English, when we speak Chinese, we have to go through a translation step before we can send it to the model, and then translate the English results returned by the model back to Chinese .

This also explains why there is a phenomenon of alternating Chinese and English in the dialogues of NPC characters.

But the ACE platform is open, and the model responsible for generating content is not irreplaceable; on the other hand, domestic manufacturers also announced their follow-up to this technology at CES.

Perhaps soon you will be able to play with NPCs in Chinese. Do you look forward to such a game?

The above is the detailed content of Actual test of NVIDIA AI game engine: real-time chat with NPC, Chinese is fluent. For more information, please follow other related articles on the PHP Chinese website!