search
HomeTechnology peripheralsAIHigh-gloss reflective terminator? Google NeRF-Casting: Ray tracing can do it!

NeRF is no longer "afraid" of near specular reflections

Early NeRF variants used multilayer perceptrons (MLPs) to map from 3D coordinates to volume density and viewpoints Related colors, but the large MLPs required to represent detailed 3D geometry and color are extremely slow to train and evaluate. Recent work has focused on making NeRF more efficient by replacing large MLPs with voxel grid-like data structures or a combination of grids and small MLPs. While it can be extended to represent detailed large-scale scenes, its advantages are limited to three-dimensional geometry and predominantly diffuse color. Expanding NeRF’s ability to model realistic viewpoint-dependent appearance remains a challenge. The current advanced model for view synthesis of shiny objects has limitations in two aspects:

can only synthesize accurate reflections of distant ambient lighting, while rendering close Poor performance in realistic reflections of scene content.

Relying on large MLPs to represent the viewpoint-related outgoing radiation of any point is difficult to scale to larger realistic scenes with detailed reflections.
  • NeRF-Casting is a method to solve these problems by introducing ray tracing into the NeRF rendering model. It mainly involves 3 areas:

Reflection modeling

: Traditional reflection modeling methods use physical laws and image-based techniques to represent surface reflection properties. In recent years, neural networks have been used to learn reflection properties, especially under complex materials and lighting conditions.
  • Ray Tracing: Ray tracing is a widely used computer graphics technique that produces realistic images by simulating the interaction of light with object surfaces. Ray tracing techniques have been used to generate high-quality reflection and refraction effects, but with high computational complexity.
  • Three-dimensional imaging: Three-dimensional imaging technology involves generating three-dimensional representations from multi-view data. NeRF and other neural network methods have made significant progress in the field of 3D imaging by learning the 3D geometry and color distribution of a scene to generate new views.
Instead of querying an expensive MLP at the points of each camera ray to obtain a viewpoint-dependent appearance, NeRF-Casting casts reflection lines from these points to In NeRF geometry, we sample correctly anti-aliased reflection in-scene content features and use a small MLP to decode these features into reflection colors. Casting rays into the recovered NeRF naturally synthesizes consistent reflections of near and far interior content. Computing appearance via ray tracing reduces the burden of representing highly detailed viewpoint-related functions with large MLPs at every point in the scene.

High-gloss reflective terminator? Google NeRF-Casting: Ray tracing can do it!Interested friends can watch the video effect: https://nerf-casting.github.io

Model details

NeRF-Casting’s Three main goals:

Want to model accurate, detailed reflections without relying on computationally expensive MLP evaluation.

Want to cast only a small amount of reflected light.
  • Hopefully minimize the amount of computation required to look up our representation at each point of these reflected rays.
  • Based on Zip-NeRF[2]: Utilize a multi-scale hash grid to store 3D features, a small MLP (1 layer, width 64) to decode these features into densities, a larger MLP (3 layers, width 256) decodes these features into colors. This means that it is relatively cheap to query the density and characteristics of a sample along a ray. Given these constraints, the following process is followed to render a specular appearance:
Query the volume density along each camera ray to calculate the ray's expected end point and surface normal.

Project a reflection cone through the desired end point in the direction of reflection.
  • Use a small MLP to combine the accumulated reflection features with other sampled quantities (such as diffuse color features and blending weights for each sample) to generate a color value for each sample along the ray.
  • Alpha composite these samples and densities to get the final color.

Reflection cone trackingHigh-gloss reflective terminator? Google NeRF-Casting: Ray tracing can do it!

High-gloss reflective terminator? Google NeRF-Casting: Ray tracing can do it!

Then construct a new reflected light direction by reflecting the initial light about the surface normal

High-gloss reflective terminator? Google NeRF-Casting: Ray tracing can do it!

High-gloss reflective terminator? Google NeRF-Casting: Ray tracing can do it!

High-gloss reflective terminator? Google NeRF-Casting: Ray tracing can do it!

High-gloss reflective terminator? Google NeRF-Casting: Ray tracing can do it!

High-gloss reflective terminator? Google NeRF-Casting: Ray tracing can do it!

High-gloss reflective terminator? Google NeRF-Casting: Ray tracing can do it!

##Conical reflection feature

Now that a vMF distribution has been defined covering the reflection rays, the goal is to estimate the expected volume rendering characteristics on the vMF distribution and then decode it into reflection colors. This expected characteristic can be written as:

High-gloss reflective terminator? Google NeRF-Casting: Ray tracing can do it!

Using Monte Carlo methods to perform integral estimates on randomly sampled rays is very expensive because each sample requires a volume along the ray render. Inspired by Zip-NeRF, this integral is approximated using a small set of representative samples combined with feature subtraction. However, unlike Zip-NeRF, we perform both operations in the two-dimensional direction domain rather than in the three-dimensional Euclidean space.

Directional Sampling

High-gloss reflective terminator? Google NeRF-Casting: Ray tracing can do it!

High-gloss reflective terminator? Google NeRF-Casting: Ray tracing can do it!

##Reflection Feature Reduction

The direction sampling described above helps select a small representative set of rays to average. However, for surfaces with high roughness, the sampled rays may be far apart relative to the underlying 3D grid cells. This means that the features in Equation 9 may be subject to artifacts, and small changes in the direction of the reflected ray may cause large changes in appearance.

In order to prevent this from happening, adjust the "feature weight reduction" technology in Zip-NeRF to the direction setting. This is achieved by multiplying features corresponding to voxels that are smaller compared to the vMF cone by a small multiplier, reducing their impact on the rendered color. According to Zip-NeRF, the weight-reducing feature at point x is defined as:

High-gloss reflective terminator? Google NeRF-Casting: Ray tracing can do it!

High-gloss reflective terminator? Google NeRF-Casting: Ray tracing can do it!

High-gloss reflective terminator? Google NeRF-Casting: Ray tracing can do it!

High-gloss reflective terminator? Google NeRF-Casting: Ray tracing can do it!##Color decoder

The role of the color decoder is to assign a color to each sampling point along the ray, using the convexity of the two color components Combination:

The first color component Cv is similar to the typical NeRF perspective-dependent appearance model: High-gloss reflective terminator? Google NeRF-Casting: Ray tracing can do it!

The second A component Cr, intended to simulate a glossy appearance, is calculated as follows: High-gloss reflective terminator? Google NeRF-Casting: Ray tracing can do it!

High-gloss reflective terminator? Google NeRF-Casting: Ray tracing can do it!

High-gloss reflective terminator? Google NeRF-Casting: Ray tracing can do it!Geometric representation and regularization

High-gloss reflective terminator? Google NeRF-Casting: Ray tracing can do it!

Appreciation of the effect

High-gloss reflective terminator? Google NeRF-Casting: Ray tracing can do it!

High-gloss reflective terminator? Google NeRF-Casting: Ray tracing can do it!

High-gloss reflective terminator? Google NeRF-Casting: Ray tracing can do it!

High-gloss reflective terminator? Google NeRF-Casting: Ray tracing can do it!

##To summarize

NeRF-Casting is a method of rendering scenes containing specular objects using Neural Radiation Fields (NeRF).

Method: Reflection cones are reflected from surfaces in the scene and tracked via NeRF, combined with a novel set of techniques to anti-alias these reflections, enabling synthesis of distant and near Accurately detailed reflections of field content that move consistently and smoothly across surfaces.

Discussion: Quantitatively outperforms existing view synthesis techniques, especially for smooth surfaces showing detailed specular reflections. Qualitative visual improvements are far outweighed by quantitative improvements in image metrics. Of particular note is the smooth and consistent motion of reflections synthesized by this method, which is more realistic than the view-dependent appearance presented by baseline methods. This shows that standard image error metrics (PSNR, SSIM, etc.) are insufficient to assess the quality of view-dependent appearance.

The above is the detailed content of High-gloss reflective terminator? Google NeRF-Casting: Ray tracing can do it!. For more information, please follow other related articles on the PHP Chinese website!

Statement
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn
Newest Annual Compilation Of The Best Prompt Engineering TechniquesNewest Annual Compilation Of The Best Prompt Engineering TechniquesApr 10, 2025 am 11:22 AM

For those of you who might be new to my column, I broadly explore the latest advances in AI across the board, including topics such as embodied AI, AI reasoning, high-tech breakthroughs in AI, prompt engineering, training of AI, fielding of AI, AI re

Europe's AI Continent Action Plan: Gigafactories, Data Labs, And Green AIEurope's AI Continent Action Plan: Gigafactories, Data Labs, And Green AIApr 10, 2025 am 11:21 AM

Europe's ambitious AI Continent Action Plan aims to establish the EU as a global leader in artificial intelligence. A key element is the creation of a network of AI gigafactories, each housing around 100,000 advanced AI chips – four times the capaci

Is Microsoft's Straightforward Agent Story Enough To Create More Fans?Is Microsoft's Straightforward Agent Story Enough To Create More Fans?Apr 10, 2025 am 11:20 AM

Microsoft's Unified Approach to AI Agent Applications: A Clear Win for Businesses Microsoft's recent announcement regarding new AI agent capabilities impressed with its clear and unified presentation. Unlike many tech announcements bogged down in te

Selling AI Strategy To Employees: Shopify CEO's ManifestoSelling AI Strategy To Employees: Shopify CEO's ManifestoApr 10, 2025 am 11:19 AM

Shopify CEO Tobi Lütke's recent memo boldly declares AI proficiency a fundamental expectation for every employee, marking a significant cultural shift within the company. This isn't a fleeting trend; it's a new operational paradigm integrated into p

IBM Launches Z17 Mainframe With Full AI IntegrationIBM Launches Z17 Mainframe With Full AI IntegrationApr 10, 2025 am 11:18 AM

IBM's z17 Mainframe: Integrating AI for Enhanced Business Operations Last month, at IBM's New York headquarters, I received a preview of the z17's capabilities. Building on the z16's success (launched in 2022 and demonstrating sustained revenue grow

5 ChatGPT Prompts To Stop Depending On Others And Trust Yourself Fully5 ChatGPT Prompts To Stop Depending On Others And Trust Yourself FullyApr 10, 2025 am 11:17 AM

Unlock unshakeable confidence and eliminate the need for external validation! These five ChatGPT prompts will guide you towards complete self-reliance and a transformative shift in self-perception. Simply copy, paste, and customize the bracketed in

AI Is Dangerously Similar To Your MindAI Is Dangerously Similar To Your MindApr 10, 2025 am 11:16 AM

A recent [study] by Anthropic, an artificial intelligence security and research company, begins to reveal the truth about these complex processes, showing a complexity that is disturbingly similar to our own cognitive domain. Natural intelligence and artificial intelligence may be more similar than we think. Snooping inside: Anthropic Interpretability Study The new findings from the research conducted by Anthropic represent significant advances in the field of mechanistic interpretability, which aims to reverse engineer internal computing of AI—not just observe what AI does, but understand how it does it at the artificial neuron level. Imagine trying to understand the brain by drawing which neurons fire when someone sees a specific object or thinks about a specific idea. A

Dragonwing Showcases Qualcomm's Edge MomentumDragonwing Showcases Qualcomm's Edge MomentumApr 10, 2025 am 11:14 AM

Qualcomm's Dragonwing: A Strategic Leap into Enterprise and Infrastructure Qualcomm is aggressively expanding its reach beyond mobile, targeting enterprise and infrastructure markets globally with its new Dragonwing brand. This isn't merely a rebran

See all articles

Hot AI Tools

Undresser.AI Undress

Undresser.AI Undress

AI-powered app for creating realistic nude photos

AI Clothes Remover

AI Clothes Remover

Online AI tool for removing clothes from photos.

Undress AI Tool

Undress AI Tool

Undress images for free

Clothoff.io

Clothoff.io

AI clothes remover

AI Hentai Generator

AI Hentai Generator

Generate AI Hentai for free.

Hot Article

R.E.P.O. Energy Crystals Explained and What They Do (Yellow Crystal)
3 weeks agoBy尊渡假赌尊渡假赌尊渡假赌
R.E.P.O. Best Graphic Settings
3 weeks agoBy尊渡假赌尊渡假赌尊渡假赌
R.E.P.O. How to Fix Audio if You Can't Hear Anyone
3 weeks agoBy尊渡假赌尊渡假赌尊渡假赌
WWE 2K25: How To Unlock Everything In MyRise
3 weeks agoBy尊渡假赌尊渡假赌尊渡假赌

Hot Tools

ZendStudio 13.5.1 Mac

ZendStudio 13.5.1 Mac

Powerful PHP integrated development environment

Atom editor mac version download

Atom editor mac version download

The most popular open source editor

Safe Exam Browser

Safe Exam Browser

Safe Exam Browser is a secure browser environment for taking online exams securely. This software turns any computer into a secure workstation. It controls access to any utility and prevents students from using unauthorized resources.

SublimeText3 Linux new version

SublimeText3 Linux new version

SublimeText3 Linux latest version

SublimeText3 Chinese version

SublimeText3 Chinese version

Chinese version, very easy to use