Zero AI content! Pure random mathematics infinitely generates realistic 3D worlds, a work by a Chinese from Princeton.

Who said that generating images and videos must rely on AI?

Princeton University’s new artifact can generate infinitely realistic 3D worlds, with special emphasis on "No AI".

Feel the effect briefly:

##Don’t think that what is generated is just a video. In fact, there is a complete set of 3D assets behind it. Built based on the modeling software Blender.

At this time, all blender users joined me: Huh?

In this way, we can use parameters to control the details:

Or get the corresponding Optical flow map, 3D scene optical flow map, depth map, panoramic segmentation map, etc., easily hold various CV tasks.

The most important thing is that it is

free and open source!

Mom really no longer has to worry that I can’t find high-quality 3D data to feed AI...

100% 3D data generator based on random mathematics

Despite the rapid development of AI, many tasks in the CV field still lack high-quality data, especially in 3D.

One solution is to use

synthetic data. It turns out that models trained on this type of data can also perform very well on real images with zero samples.

But the problem is that most existing free 3D synthetic data tools are basically limited to a single scene: either autonomous driving related, or man-made objects located in indoor environments.

Therefore, in order to expand the coverage, especially natural scenes in the real world, the author created this based on Blender

Based on random mathematical rulesInfinitely generate various scenes Infinigen.

Infinigen mainly uses Blender's "primitives"

(or primitives) to design a program rule library to complete various aspects of real natural scenes through coding. Generation of objects.

The paper mainly introduces Infinigen's program system, including:

Node Transpiler
• (node ​​converter), which can automatically convert the Blender node graph into Python code , making it easier for non-programmer users to use Infinigen.

As shown in the figure below, the code it generates is more general, allowing us to randomize both the input parameters and the graph structure.

Generator Subsystems
• (generator subsystem), Infinigen’s generator is a probability-based program, each program is specialized Used to generate a sub-object (such as a mountain or a fish) . Each object has a set of high-level parameters (such as the total height of a mountain) that users can adjust using the Python API to achieve fine-grained control over data generation.
Material Generators
• (Material Generator), 50 in total, each consisting of a random shader that can specify color and reflectivity and a local geometry that generates corresponding fine geometric details Generator composition.

As shown in the figure below, it can ensure very realistic geometric details.

##Terrain Generators
• (Terrain Generator), as shown in the figure below, this generator can be squeezed repeatedly Press to generate boulders, and use Blender's built-in plug-in to generate small rocks.

#And helped Infinigen simulate dynamic fluids using FLIP and weather using Blender’s particle system.

Plants & Underwater Object Generators
• (Plants and Underwater Object Generators), including using algorithms such as random walks to grow trees Model to create a 3D world covered in various trees, shrubs and even cacti.

Or you can use differential growth, Laplacian growth and reaction diffusion to create various corals, and use geometric node graphs to generate leaves, flowers, seaweed, Kelp, molluscs and jellyfish.

There are also various sub-generators

(such as biological generators)I will not introduce them one by one.

In addition to these, Infinigen also includes an image rendering and Ground Truth extraction program, which is mainly used to generate the types of images shown below.

For the former, the system uses Blender's path tracing renderer Cycles based on natural laws to render the image.

The author introduced that although Blender was used to develop Infinigen's program rules, a large part of the program generation was completed outside Blender.

In addition, they also stated that building Infinigen was an extremely large software project, and the main branch of its code base alone included

40485 lines of code.

Finally, Infinigen was benchmarked on 2 Intel Xeon Silver 4114 @ 2.20GHz CPUs and 1 NVidia GPU, generating a pair of 1080p images in

(wall time) ##3.5 hours. The table below is how it compares to existing synthetic datasets or generators.

The author said that it can be seen that the biggest advantage of Infinigen is that

it can programmatically generate

without any external reference library Unlimited natural 3D data, nothing else will work. Team Introduction

Infinigen will conduct a poster presentation at the CVPR conference this Wednesday.

Its authors are all from the Vision & Learning Lab of Princeton University.

Three co-authors, one of whom is

Ma Zeyu, is a third-year PhD student at Princeton University and graduated from Tsinghua University in 2020 with a bachelor's degree in electrical engineering. The corresponding author is Associate Professor of Computer Science Department of Princeton University

Deng Jia.

Currently, Infinigen’s code has been launched on GitHub, with 850 stars in just two days.

Paper address:
##https://www.php. cn/link/d9812f756d0df06c7381945d2e2c7d4b##Project homepage:
##https://www.php.cn/link/ca886eb9edb61a42256192745c72cd79
##GitHub address:
##https://www.php.cn/link/a1cb608a30fc2883eed0831dcf25f260

The above is the detailed content of Zero AI content! Pure random mathematics infinitely generates realistic 3D worlds, a work by a Chinese from Princeton.. For more information, please follow other related articles on the PHP Chinese website!