2022 Taobao Creation Festival 3D Live Virtual Camp Technology Highlights Revealed

On August 24, 2022, Taobao Maker Festival will debut! In order to explore the next generation of Internet immersive shopping experience, in addition to building a 20,000-square-meter panoramic "Future Camp" in the offline Canton Fair exhibition hall, this Creation Festival also launched online for the first time a virtual interactive space "3D Crazy" that can accommodate tens of thousands of people online at the same time. City-Creation Festival Virtual Camp”.

In the 3D virtual camp, users can experience high-definition game textures in a lightweight way, and join the world of the "3D People and Goods Yard" as virtual characters, interacting with players and 3D products. Satisfy the demands of the new generation of consumer groups who love to play and create, and bring a new consumer experience with innovative technology.

Let’s take a look at the experience & technical highlights of the 3D live virtual camp:

Lightweight naked-eye 3D “real game”

Once you liked a 3D game, the client downloaded several gigabytes, but you had to uninstall it when you wanted to play a new game. It took up too much memory! In the virtual camp of the Creation Festival, you can quickly load a naked-eye 3D, interactive, live-streaming, high-definition “game” that you can even buy, and your Taobao App “hasn’t gotten any bigger at all.” Loading on your phone The speed is "whooshing"~

▐ Technical Highlights: Cloud Cost Challenge

Traditional development of 3D game applications on the mobile side often requires Integrating dozens or hundreds of megabytes of game engines in end testing requires downloading several gigabytes of material resources, which is currently unrealistic in the Taobao app. Using cloud rendering can solve this problem very well. With the help of powerful GPU in the cloud, users can complete high-definition and complex scene rendering. Users do not need to download large resource files or install any content. The only thing required is to play real-time rendered media. flow.

But the biggest problem here is the cost of cloud rendering machines. In order to reduce the cost of cloud rendering machines, in addition to cloud rendering, we also carry out performance optimization strategies such as model surface reduction, DP merging, texture optimization, and video material resolution reduction. In terms of machine scheduling, a time-based dynamic expansion and contraction strategy was developed based on business scenarios to maximize machine utilization.

3D people and 3D products meet in the 3D virtual world for the first time

The 3D users who once dressed up in "Taobao Life" "walked" out of their small homes for the first time and came to Creation Festival’s live 3D virtual camp. Here, you can jump freely, light bonfires, view 3D products, watch sellers’ live broadcasts, and interact with players. 3D people, 3D products, and 3D environments are combined to give you an immersive and wonderful camp adventure experience.

▐ Technical highlights: Construction and interaction of people and goods yard

Interaction between people. Placing the 3D characters in Taobao Life into the 3D virtual world first requires certain asset conversion and rendering costs. The user's running around the camp involves real-time synchronization of the user's location. For users' real-time roaming, we use frame synchronization technology to update user location information at a fixed frequency. Ideally, users will not feel stuck, but network congestion inevitably causes inter-frame jitter. If the synchronization time exceeds a certain interval, it will cause a jump. In order to make up for this shortcoming, we use a motion compensation algorithm when rendering characters. Simply put, it uses algorithm simulation to make the transition of user position movement smoother. At the same time, in terms of synchronization of interactive data, the AOI grid algorithm is adopted to solve the problem of data synchronization of multiple people on the same screen. In this way, 3D characters will run more smoothly in the virtual world.

Interaction between people and products. It is not easy for users to freely view the details of 3D products in a virtual scene. We have two options to choose from: one is mobile rendering and the other is cloud rendering. Mobile rendering also has relatively mature solutions in Taobao. Unfortunately, the mobile rendering engine and the cloud rendering engine are not the same set, so the same product model cannot be used universally. At the same time, dynamic downloading of the model, surface reduction optimization, etc. must also be considered. Problem, in order to avoid these disadvantages, we still chose the cloud rendering solution. By dynamically switching the camera perspective and responding to the user's interactive instructions in real time, we can achieve the experience of rotating and zooming to watch 3D products.

1080P resolution and cool high-definition quality

Once you tried the page mini-game in the mobile app, and it was as blurry as a mosaic. You ran and jumped and dropped frames. The Creation Festival’s new 3D virtual camp features high-definition images, smooth effects, and the stage lighting can dynamically track you as a “star”~

▐ Technical highlights: content production and real-time transmission of cloud rendering

In the cloud, we use Unreal Engine for real-time picture rendering, and combine it with dynamic camera movement, dynamic stage lighting, multi-view characters, and particle effects. Technologies such as this make the picture look cooler and provide a stronger sense of user interaction. In order to transmit high-definition images to users' mobile phones, we need to comprehensively consider the balance of image quality, lagging, and delay. With the help of the GRTN transmission network jointly built by Taobao and Alibaba Cloud, as well as self-developed encoding and decoding algorithms, we can maximize Ensure the clarity of the picture quality.

Interactive virtual screen does not freeze

Once you were in the live broadcast room of a Taobao anchor, counting down "3, 2, 1, start grabbing!" and felt the thrill of ultra-low latency flash sales. But you outside the screen cannot directly intervene in the actions of the host in the live broadcast room through interaction. In the 3D virtual camp, you can dance and say hello, click on the merchants' 3D products, and watch the explanations of your favorite anchors. You can truly "communicate with the merchants and anchors in the same frame."

▐ Technical highlights: Full-link ultra-low latency

Different from non-interfering picture transmission, ultra-low latency and smoothness are achieved interactively For experience, we need to complete the upstreaming of user instructions, the rendering of real-time images, and return the rendered images to the user’s mobile phone within a hundred milliseconds. First of all, it is necessary to comprehensively consider the user's machine, network, and access point conditions to dynamically select the streaming node closest to the user, thereby adjusting the cloud streaming strategy (including definition and encoding method selection). Secondly, during the transmission process, weak network control strategies such as real-time bandwidth feedback, FEC, and dynamic buffers need to be used to combat network jitter, so that the entire link delay reaches a reasonable position, and ultimately achieve interaction in the virtual world. Ensure a smooth interactive experience.

Conclusion

This Creation Festival 3D virtual camp is the first online attempt of Taobao’s virtual interactive space “3D Crazy City” series. In the next step, we will combine cloud real-time rendering and XR/CG technology to realize virtual scene customization, support larger-scale user real-time interaction, and bring a more immersive interactive experience to consumers.

The above is the detailed content of 2022 Taobao Creation Festival 3D Live Virtual Camp Technology Highlights Revealed. For more information, please follow other related articles on the PHP Chinese website!