search
HomeTechnology peripheralsAIHigh-gloss reflective terminator? Google NeRF-Casting: Ray tracing can do it!

NeRF is no longer "afraid" of near specular reflections

Early NeRF variants used multilayer perceptrons (MLPs) to map from 3D coordinates to volume density and viewpoints Related colors, but the large MLPs required to represent detailed 3D geometry and color are extremely slow to train and evaluate. Recent work has focused on making NeRF more efficient by replacing large MLPs with voxel grid-like data structures or a combination of grids and small MLPs. While it can be extended to represent detailed large-scale scenes, its advantages are limited to three-dimensional geometry and predominantly diffuse color. Expanding NeRF’s ability to model realistic viewpoint-dependent appearance remains a challenge. The current advanced model for view synthesis of shiny objects has limitations in two aspects:

can only synthesize accurate reflections of distant ambient lighting, while rendering close Poor performance in realistic reflections of scene content.

Relying on large MLPs to represent the viewpoint-related outgoing radiation of any point is difficult to scale to larger realistic scenes with detailed reflections.
  • NeRF-Casting is a method to solve these problems by introducing ray tracing into the NeRF rendering model. It mainly involves 3 areas:

Reflection modeling

: Traditional reflection modeling methods use physical laws and image-based techniques to represent surface reflection properties. In recent years, neural networks have been used to learn reflection properties, especially under complex materials and lighting conditions.
  • Ray Tracing: Ray tracing is a widely used computer graphics technique that produces realistic images by simulating the interaction of light with object surfaces. Ray tracing techniques have been used to generate high-quality reflection and refraction effects, but with high computational complexity.
  • Three-dimensional imaging: Three-dimensional imaging technology involves generating three-dimensional representations from multi-view data. NeRF and other neural network methods have made significant progress in the field of 3D imaging by learning the 3D geometry and color distribution of a scene to generate new views.
Instead of querying an expensive MLP at the points of each camera ray to obtain a viewpoint-dependent appearance, NeRF-Casting casts reflection lines from these points to In NeRF geometry, we sample correctly anti-aliased reflection in-scene content features and use a small MLP to decode these features into reflection colors. Casting rays into the recovered NeRF naturally synthesizes consistent reflections of near and far interior content. Computing appearance via ray tracing reduces the burden of representing highly detailed viewpoint-related functions with large MLPs at every point in the scene.

High-gloss reflective terminator? Google NeRF-Casting: Ray tracing can do it!Interested friends can watch the video effect: https://nerf-casting.github.io

Model details

NeRF-Casting’s Three main goals:

Want to model accurate, detailed reflections without relying on computationally expensive MLP evaluation.

Want to cast only a small amount of reflected light.
  • Hopefully minimize the amount of computation required to look up our representation at each point of these reflected rays.
  • Based on Zip-NeRF[2]: Utilize a multi-scale hash grid to store 3D features, a small MLP (1 layer, width 64) to decode these features into densities, a larger MLP (3 layers, width 256) decodes these features into colors. This means that it is relatively cheap to query the density and characteristics of a sample along a ray. Given these constraints, the following process is followed to render a specular appearance:
Query the volume density along each camera ray to calculate the ray's expected end point and surface normal.

Project a reflection cone through the desired end point in the direction of reflection.
  • Use a small MLP to combine the accumulated reflection features with other sampled quantities (such as diffuse color features and blending weights for each sample) to generate a color value for each sample along the ray.
  • Alpha composite these samples and densities to get the final color.

Reflection cone trackingHigh-gloss reflective terminator? Google NeRF-Casting: Ray tracing can do it!

High-gloss reflective terminator? Google NeRF-Casting: Ray tracing can do it!

Then construct a new reflected light direction by reflecting the initial light about the surface normal

High-gloss reflective terminator? Google NeRF-Casting: Ray tracing can do it!

High-gloss reflective terminator? Google NeRF-Casting: Ray tracing can do it!

High-gloss reflective terminator? Google NeRF-Casting: Ray tracing can do it!

High-gloss reflective terminator? Google NeRF-Casting: Ray tracing can do it!

High-gloss reflective terminator? Google NeRF-Casting: Ray tracing can do it!

High-gloss reflective terminator? Google NeRF-Casting: Ray tracing can do it!

##Conical reflection feature

Now that a vMF distribution has been defined covering the reflection rays, the goal is to estimate the expected volume rendering characteristics on the vMF distribution and then decode it into reflection colors. This expected characteristic can be written as:

High-gloss reflective terminator? Google NeRF-Casting: Ray tracing can do it!

Using Monte Carlo methods to perform integral estimates on randomly sampled rays is very expensive because each sample requires a volume along the ray render. Inspired by Zip-NeRF, this integral is approximated using a small set of representative samples combined with feature subtraction. However, unlike Zip-NeRF, we perform both operations in the two-dimensional direction domain rather than in the three-dimensional Euclidean space.

Directional Sampling

High-gloss reflective terminator? Google NeRF-Casting: Ray tracing can do it!

High-gloss reflective terminator? Google NeRF-Casting: Ray tracing can do it!

##Reflection Feature Reduction

The direction sampling described above helps select a small representative set of rays to average. However, for surfaces with high roughness, the sampled rays may be far apart relative to the underlying 3D grid cells. This means that the features in Equation 9 may be subject to artifacts, and small changes in the direction of the reflected ray may cause large changes in appearance.

In order to prevent this from happening, adjust the "feature weight reduction" technology in Zip-NeRF to the direction setting. This is achieved by multiplying features corresponding to voxels that are smaller compared to the vMF cone by a small multiplier, reducing their impact on the rendered color. According to Zip-NeRF, the weight-reducing feature at point x is defined as:

High-gloss reflective terminator? Google NeRF-Casting: Ray tracing can do it!

High-gloss reflective terminator? Google NeRF-Casting: Ray tracing can do it!

High-gloss reflective terminator? Google NeRF-Casting: Ray tracing can do it!

High-gloss reflective terminator? Google NeRF-Casting: Ray tracing can do it!##Color decoder

The role of the color decoder is to assign a color to each sampling point along the ray, using the convexity of the two color components Combination:

The first color component Cv is similar to the typical NeRF perspective-dependent appearance model: High-gloss reflective terminator? Google NeRF-Casting: Ray tracing can do it!

The second A component Cr, intended to simulate a glossy appearance, is calculated as follows: High-gloss reflective terminator? Google NeRF-Casting: Ray tracing can do it!

High-gloss reflective terminator? Google NeRF-Casting: Ray tracing can do it!

High-gloss reflective terminator? Google NeRF-Casting: Ray tracing can do it!Geometric representation and regularization

High-gloss reflective terminator? Google NeRF-Casting: Ray tracing can do it!

Appreciation of the effect

High-gloss reflective terminator? Google NeRF-Casting: Ray tracing can do it!

High-gloss reflective terminator? Google NeRF-Casting: Ray tracing can do it!

High-gloss reflective terminator? Google NeRF-Casting: Ray tracing can do it!

High-gloss reflective terminator? Google NeRF-Casting: Ray tracing can do it!

##To summarize

NeRF-Casting is a method of rendering scenes containing specular objects using Neural Radiation Fields (NeRF).

Method: Reflection cones are reflected from surfaces in the scene and tracked via NeRF, combined with a novel set of techniques to anti-alias these reflections, enabling synthesis of distant and near Accurately detailed reflections of field content that move consistently and smoothly across surfaces.

Discussion: Quantitatively outperforms existing view synthesis techniques, especially for smooth surfaces showing detailed specular reflections. Qualitative visual improvements are far outweighed by quantitative improvements in image metrics. Of particular note is the smooth and consistent motion of reflections synthesized by this method, which is more realistic than the view-dependent appearance presented by baseline methods. This shows that standard image error metrics (PSNR, SSIM, etc.) are insufficient to assess the quality of view-dependent appearance.

The above is the detailed content of High-gloss reflective terminator? Google NeRF-Casting: Ray tracing can do it!. For more information, please follow other related articles on the PHP Chinese website!

Statement
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn
谷歌三件套指的是哪三个软件谷歌三件套指的是哪三个软件Sep 30, 2022 pm 01:54 PM

谷歌三件套指的是:1、google play商店,即下载各种应用程序的平台,类似于移动助手,安卓用户可以在商店下载免费或付费的游戏和软件;2、Google Play服务,用于更新Google本家的应用和Google Play提供的其他第三方应用;3、谷歌服务框架(GMS),是系统软件里面可以删除的一个APK程序,通过谷歌平台上架的应用和游戏都需要框架的支持。

为什么中国不卖google手机为什么中国不卖google手机Mar 30, 2023 pm 05:31 PM

中国不卖google手机的原因:谷歌已经全面退出中国市场了,所以不能在中国销售,在国内是没有合法途径销售。在中国消费市场中,消费者大都倾向于物美价廉以及功能实用的产品,所以竞争实力本就因政治因素大打折扣的谷歌手机主体市场一直不在中国大陆。

谷歌超强AI超算碾压英伟达A100!TPU v4性能提升10倍,细节首次公开谷歌超强AI超算碾压英伟达A100!TPU v4性能提升10倍,细节首次公开Apr 07, 2023 pm 02:54 PM

虽然谷歌早在2020年,就在自家的数据中心上部署了当时最强的AI芯片——TPU v4。但直到今年的4月4日,谷歌才首次公布了这台AI超算的技术细节。论文地址:https://arxiv.org/abs/2304.01433相比于TPU v3,TPU v4的性能要高出2.1倍,而在整合4096个芯片之后,超算的性能更是提升了10倍。另外,谷歌还声称,自家芯片要比英伟达A100更快、更节能。与A100对打,速度快1.7倍论文中,谷歌表示,对于规模相当的系统,TPU v4可以提供比英伟达A100强1.

LLM之战,谷歌输了!越来越多顶尖研究员跳槽OpenAILLM之战,谷歌输了!越来越多顶尖研究员跳槽OpenAIApr 07, 2023 pm 05:48 PM

​前几天,谷歌差点遭遇一场公关危机,Bert一作、已跳槽OpenAI的前员工Jacob Devlin曝出,Bard竟是用ChatGPT的数据训练的。随后,谷歌火速否认。而这场争议,也牵出了一场大讨论:为什么越来越多Google顶尖研究员跳槽OpenAI?这场LLM战役它还能打赢吗?知友回复莱斯大学博士、知友「一堆废纸」表示,其实谷歌和OpenAI的差距,是数据的差距。「OpenAI对LLM有强大的执念,这是Google这类公司完全比不上的。当然人的差距只是一个方面,数据的差距以及对待数据的态度才

谷歌并未放弃TensorFlow,将于2023年发布新版,明确四大支柱谷歌并未放弃TensorFlow,将于2023年发布新版,明确四大支柱Apr 12, 2023 am 11:52 AM

2015 年,谷歌大脑开放了一个名为「TensorFlow」的研究项目,这款产品迅速流行起来,成为人工智能业界的主流深度学习框架,塑造了现代机器学习的生态系统。从那时起,成千上万的开源贡献者以及众多的开发人员、社区组织者、研究人员和教育工作者等都投入到这一开源软件库上。然而七年后的今天,故事的走向已经完全不同:谷歌的 TensorFlow 失去了开发者的拥护。因为 TensorFlow 用户已经开始转向 Meta 推出的另一款框架 PyTorch。众多开发者都认为 TensorFlow 已经输掉

四分钟对打300多次,谷歌教会机器人打乒乓球四分钟对打300多次,谷歌教会机器人打乒乓球Apr 10, 2023 am 09:11 AM

让一位乒乓球爱好者和机器人对打,按照机器人的发展趋势来看,谁输谁赢还真说不准。​机器人拥有灵巧的可操作性、腿部运动灵活、抓握能力出色…… 已被广泛应用于各种挑战任务。但在与人类互动紧密的任务中,机器人的表现又如何呢?就拿乒乓球来说,这需要双方高度配合,并且球的运动非常快速,这对算法提出了重大挑战。在乒乓球比赛中,首要的就是速度和精度,这对学习算法提出了很高的要求。同时,这项运动具有高度结构化(具有固定的、可预测的环境)和多智能体协作(机器人可以与人类或其他机器人一起对打)两大特点,使其成为研究人

参数少量提升,性能指数爆发!谷歌:大语言模型暗藏「神秘技能」参数少量提升,性能指数爆发!谷歌:大语言模型暗藏「神秘技能」Apr 11, 2023 pm 11:16 PM

由于可以做一些没训练过的事情,大型语言模型似乎具有某种魔力,也因此成为了媒体和研究员炒作和关注的焦点。当扩展大型语言模型时,偶尔会出现一些较小模型没有的新能力,这种类似于「创造力」的属性被称作「突现」能力,代表我们向通用人工智能迈进了一大步。如今,来自谷歌、斯坦福、Deepmind和北卡罗来纳大学的研究人员,正在探索大型语言模型中的「突现」能力。解码器提示的 DALL-E神奇的「突现」能力自然语言处理(NLP)已经被基于大量文本数据训练的语言模型彻底改变。扩大语言模型的规模通常会提高一系列下游N

超5800亿美元!微软谷歌神仙打架,让英伟达市值飙升,约为5个英特尔超5800亿美元!微软谷歌神仙打架,让英伟达市值飙升,约为5个英特尔Apr 11, 2023 pm 04:31 PM

ChatGPT在手,有问必答。你可知,与它每次对话的计算成本简直让人泪目。此前,分析师称ChatGPT回复一次,需要2美分。要知道,人工智能聊天机器人所需的算力背后烧的可是GPU。这恰恰让像英伟达这样的芯片公司豪赚了一把。2月23日,英伟达股价飙升,使其市值增加了700多亿美元,总市值超5800亿美元,大约是英特尔的5倍。在英伟达之外,AMD可以称得上是图形处理器行业的第二大厂商,市场份额约为20%。而英特尔持有不到1%的市场份额。ChatGPT在跑,英伟达在赚随着ChatGPT解锁潜在的应用案

See all articles

Hot AI Tools

Undresser.AI Undress

Undresser.AI Undress

AI-powered app for creating realistic nude photos

AI Clothes Remover

AI Clothes Remover

Online AI tool for removing clothes from photos.

Undress AI Tool

Undress AI Tool

Undress images for free

Clothoff.io

Clothoff.io

AI clothes remover

AI Hentai Generator

AI Hentai Generator

Generate AI Hentai for free.

Hot Article

R.E.P.O. Energy Crystals Explained and What They Do (Yellow Crystal)
2 weeks agoBy尊渡假赌尊渡假赌尊渡假赌
Repo: How To Revive Teammates
4 weeks agoBy尊渡假赌尊渡假赌尊渡假赌
Hello Kitty Island Adventure: How To Get Giant Seeds
4 weeks agoBy尊渡假赌尊渡假赌尊渡假赌

Hot Tools

Dreamweaver CS6

Dreamweaver CS6

Visual web development tools

SecLists

SecLists

SecLists is the ultimate security tester's companion. It is a collection of various types of lists that are frequently used during security assessments, all in one place. SecLists helps make security testing more efficient and productive by conveniently providing all the lists a security tester might need. List types include usernames, passwords, URLs, fuzzing payloads, sensitive data patterns, web shells, and more. The tester can simply pull this repository onto a new test machine and he will have access to every type of list he needs.

Safe Exam Browser

Safe Exam Browser

Safe Exam Browser is a secure browser environment for taking online exams securely. This software turns any computer into a secure workstation. It controls access to any utility and prevents students from using unauthorized resources.

EditPlus Chinese cracked version

EditPlus Chinese cracked version

Small size, syntax highlighting, does not support code prompt function

mPDF

mPDF

mPDF is a PHP library that can generate PDF files from UTF-8 encoded HTML. The original author, Ian Back, wrote mPDF to output PDF files "on the fly" from his website and handle different languages. It is slower than original scripts like HTML2FPDF and produces larger files when using Unicode fonts, but supports CSS styles etc. and has a lot of enhancements. Supports almost all languages, including RTL (Arabic and Hebrew) and CJK (Chinese, Japanese and Korean). Supports nested block-level elements (such as P, DIV),