search
HomeTechnology peripheralsAIIs it expected to replace Deepfake? Revealing how awesome this year's most popular NeRF technology is

What, you don’t know NeRF yet?

As the hottest AI technology in the field of computer vision this year, NeRF can be said to be widely used and has a bright future.

The friends on Station B have used this technology in new ways.

Is it expected to replace Deepfake? Revealing how awesome this years most popular NeRF technology is

To attract good news

So, what exactly is NeRF?

NeRF (Neural Radiance Fields) is a concept first proposed in the best paper at the 2020 ECCV conference. It pushes implicit expression to a new level, using only 2D posed images as supervision , which can represent complex three-dimensional scenes.

One stone stirred up a thousand waves. Since then, NeRF has developed rapidly and been applied to many technical directions, such as "new viewpoint synthesis, three-dimensional reconstruction", etc.

NeRF inputs sparse multi-angle images with poses for training to obtain a neural radiation field model. According to this model, clear photos from any viewing angle can be rendered, as shown in the figure below. It can also be briefly summarized as using an MLP to implicitly learn a three-dimensional scene.

Is it expected to replace Deepfake? Revealing how awesome this years most popular NeRF technology is

#Netizens will naturally compare NeRF with the equally popular Deepfake.

A recent article published by MetaPhysics took stock of the evolutionary history, challenges and advantages of NeRF, and predicted that NeRF will eventually replace Deepfake.

Most of the eye-catching topics about deepfake technology refer to the two open source software packages that have become popular since deepfakes entered the public eye in 2017: DeepFaceLab (DFL) and FaceSwap.

Is it expected to replace Deepfake? Revealing how awesome this years most popular NeRF technology is

While both packages have extensive user bases and active developer communities, neither project deviates significantly from the GitHub code.

Of course, the developers of DFL and FaceSwap have not been idle: it is now possible to train deepfake models using larger input images, although this requires more expensive GPUs.

Is it expected to replace Deepfake? Revealing how awesome this years most popular NeRF technology is

#But in fact, in the past three years, the improvement in deepfake image quality promoted by the media is mainly due to end users.

They have accumulated "time-saving and rare" experience in data collection, as well as the best methods to train models (sometimes a single experiment can take weeks), and learned how to leverage and extend the original 2017 code. The outermost limit.

Some in the VFX and ML research communities are trying to break through the "hard limits" of the popular deepfake package by extending the architecture so that machine learning models can be trained on images up to 1024×1024.

Is it expected to replace Deepfake? Revealing how awesome this years most popular NeRF technology is

The pixels are twice the current actual range of DeepFaceLab or FaceSwap, closer to the resolutions useful in film and television production.

Let’s learn about NeRF together~

Unveiling

NeRF (Neural Radiance Fields), which appeared in 2020, is a method that passes through the neural A method of reconstructing objects and environments by splicing photos from multiple viewpoints within the network.

It achieves the best results for synthesizing complex scene views by optimizing the underlying continuous volumetric scene function using a sparse set of input views.

The algorithm also uses a fully connected deep network to represent a scene, its input is a single continuous 5D coordinate (spatial position (x, y, z) and viewing direction (θ, φ)), and its output is the Volumetric density at a spatial location and associated emission amplitude brightness.

The view is synthesized by querying 5D coordinates along the camera ray, and using classic volume rendering techniques to project the output color and density into the image.

Implementation process:

Is it expected to replace Deepfake? Revealing how awesome this years most popular NeRF technology is

First represent a continuous scene as a 5D vector value function, whose input is a 3D position and 2D viewing direction, corresponding to The output of is an emission color c and volume density σ.

In practice, the 3D Cartesian unit vector d is used to represent the direction. This continuous 5D scene representation is approximated with an MLP network and its weights are optimized.

Additionally, the representation is encouraged to be consistent across multiple views by restricting the network to predict volume density σ as a function of position x, while also allowing RGB color c to be predicted as a function of position and viewing direction.

To achieve this, the MLP first processes the input 3D coordinates x with 8 fully connected layers (using ReLU activation and 256 channels per layer), and outputs σ and 256-dimensional feature vectors.

This feature vector is then concatenated with the viewing direction of the camera ray and passed to an additional fully connected layer that outputs the view-dependent RGB color.

In addition, NeRF also introduces two improvements to achieve the representation of high-resolution complex scenes. The first is positional encoding to help MLP represent high-frequency functions, and the second is a stratified sampling process to enable it to efficiently sample high-frequency representations.

As we all know, the position encoding in the Transformer architecture can provide the discrete position of the mark in the sequence as the input of the entire architecture. NeRF uses position coding to map continuous input coordinates to a higher dimensional space, making it easier for MLP to approximate higher frequency functions.

Is it expected to replace Deepfake? Revealing how awesome this years most popular NeRF technology is

As can be observed from the figure, removing positional encoding will greatly reduce the model's ability to represent high-frequency geometry and texture, ultimately leading to an over-smooth appearance.

Since the rendering strategy of densely evaluating the neural radiation field network at N query points along each camera ray is very inefficient, NeRF finally adopted a hierarchical representation, by proportioning the expected effect of the final rendering Allocate samples to improve rendering efficiency.

In short, NeRF no longer uses only one network to represent the scene, but optimizes two networks at the same time, a "coarse-grained" network and a "fine-grained" network.

The future is promising

NeRF solves the shortcomings of the past, that is, using MLP to represent objects and scenes as continuous functions. Compared with previous methods, NeRF can produce better rendering effects.

However, NeRF also faces many technical bottlenecks. For example, NeRF's accelerator will sacrifice other relatively useful functions (such as flexibility) to achieve low latency, more interactive environments, and less training time.

So, although NeRF is a key breakthrough, it still takes a certain amount of time to achieve perfect results.

Technology is progressing, and the future is still promising! ​

The above is the detailed content of Is it expected to replace Deepfake? Revealing how awesome this year's most popular NeRF technology is. For more information, please follow other related articles on the PHP Chinese website!

Statement
This article is reproduced at:51CTO.COM. If there is any infringement, please contact admin@php.cn delete
一个适合程序员的 AI创业思路一个适合程序员的 AI创业思路Apr 09, 2024 am 09:01 AM

大家好,我卡颂。许多程序员朋友都希望参与自己的AI产品开发。我们可以根据"流程自动化程度"和"AI应用程度"将产品的形态划分为四个象限。其中:流程自动化程度衡量「产品的服务流程有多少需要人工介入」AI应用程度衡量「AI在产品中应用的比重」首先,限制AI的能力,以处理一张AI图片应用,用户在应用内通过与UI交互就能完成完整的服务流程,从而自动化程度高。同时,“AI图片处理”重度依赖AI的能力,所以AI应用程度高。第二象限,是常规的应用开发领域,比如开发个知识管理应用、时间管理应用、流程自动化程度高

家电行业观察:AI加持下,全屋智能将成为智能家电未来?家电行业观察:AI加持下,全屋智能将成为智能家电未来?Jun 13, 2023 pm 05:48 PM

若是将人工智能比喻为第四次工业革命的话,那么大模型便是第四次工业革命的粮食储备。在应用层面,它使工业界得以重温1956年美国达特茅斯会议的设想,并正式开启了重塑世界的序幕。根据大厂定义,AI家电是具备互联互通、人机交互和主动决策能力的家电,AI家电可以视作是智能家电的最高形态。然而,目前市面上AI加持的全屋智能模式是否能成为未来行业的主角?家电行业是否会诞生新竞争格局?本文将从三个方面来解析。全屋智能为何雷声大,雨点小?资料来源:Statista、中安网、艾瑞咨询、洛图科技、全国制锁行业信息中心

生成式AI技术为制造企业降本增效提供强大支持生成式AI技术为制造企业降本增效提供强大支持Nov 21, 2023 am 09:13 AM

在2023年,生成式人工智能(ArtificialIntelligenceGeneratedContent,简称AIGC)成为科技领域最热门的话题,毫无疑问那么对于制造行业来说,他们应该怎样从生成式AI这项新兴技术中获益?广大正在实施数字化转型的中小企业,又可以由此获得怎样的启示?最近,亚马逊云科技与制造行业的代表一同合作,就中国制造业目前的发展趋势、传统制造业数字化转型所面临的挑战与机遇,以及生成式人工智能对制造业的创新重塑等话题进行了分享和深入探讨生成式AI在制造行业的应用现状提及中国制造业

华为余承东表示:鸿蒙可能拥有强大的人工智能大模型能力华为余承东表示:鸿蒙可能拥有强大的人工智能大模型能力Aug 04, 2023 pm 04:25 PM

华为常务董事余承东在今天的微博上发布了HDC大会邀请函,暗示鸿蒙或许将具备AI大模型能力。据他后续微博内容显示,邀请函文字是由智慧语音助手小艺生成的。余承东表示,鸿蒙世界即将带来更智能、更贴心的全新体验根据之前曝光的信息来看,今年鸿蒙4在AI能力方面有望取得重大进展,进一步巩固了AI作为鸿蒙系统的核心特性

AI客服替代优势尽显 需求匹配与普及应用尚待时日AI客服替代优势尽显 需求匹配与普及应用尚待时日Apr 12, 2023 pm 07:34 PM

​从人工呼叫中心时代,经历了IVR流程设计、在线客服系统等的应用,到已经发展至如今的人工智能(AI)客服。作为服务客户的重要窗口,客服行业始终站在时代前端,不断利用新科技发展新生产力,向着高效率化、高品质化、高服务化以及个性化、全天候客户服务迈进。伴随着客户人群、数量的增多,以及人工服务成本的快速增加,如何利用人工智能、大数据等新一代信息科技,促进各行业的客户服务中心从劳动密集型向智能化、精细化、精细化的技术转型升级,已成为摆在诸多行业面前的重要问题。得益于人工智能技术不断进步与场景化应用的快速

AI技术加速迭代:周鸿祎视角下的大模型战略AI技术加速迭代:周鸿祎视角下的大模型战略Jun 15, 2023 pm 02:25 PM

今年以来,360集团创始人周鸿祎在所有公开场合的讲话都离不开一个话题,那就是人工智能大模型。他曾自称“GPT的布道者”,对ChatGPT取得的突破赞不绝口,更是坚定看好由此产生的AI技术迭代。作为一个擅于表达的明星企业家,周鸿祎的演讲往往妙语连珠,所以他的“布道”也创造过很多热点话题,确实为AI大模型添了一把火。但对周鸿祎而言,光做意见领袖还不够,外界更关心他执掌的360公司如何应对这波AI新浪潮。事实上,在360内部,周鸿祎也早已掀起一场全员变革,4月份,他发出内部信,要求360每一位员工、每

IMAX中国AI艺术大片把影院搬到经典地标IMAX中国AI艺术大片把影院搬到经典地标Jun 10, 2023 pm 01:03 PM

IMAX中国AI艺术大片把影院搬到经典地标漓江时光网讯近日,IMAX打造中国首款AI艺术大片,在AI技术的助力下,IMAX影院“落地”包括长城、敦煌、桂林漓江、张掖丹霞在内的多个国内经典地标。此款AI艺术大片由IMAX联合数字艺术家@kefan404和尼欧数字创作,组画共四张,IMAX标志性的超大银幕或铺展于张掖丹霞缤纷绚丽的大自然“画布”之中,或于承载千年文化积淀的敦煌比邻矗立,或与桂林漓江的山水长卷融为一体,或在层峦叠嶂之中眺望巍峨长城,令人不禁期待想象成真的那一天。自2008年于日本东京巨

为什么说信息化平权是AI给人类最重要的积极意义?探索未来的故事为什么说信息化平权是AI给人类最重要的积极意义?探索未来的故事Sep 21, 2023 pm 06:21 PM

在一个充满未来科技的世界中,人工智能已经成为人类生活中必不可少的助手。然而,人工智能不仅仅是为了方便我们的生活,它还以一种悄然存在的方式改变着人类社会的结构和运行。其中最重要的积极意义之一就是信息平等化消除数字鸿沟,让每个人都能平等享受科技带来的便利在当前数字化时代,信息化已经成为推动社会发展的重要力量。然而,我们也面临着一个现实问题,即存在着数字鸿沟,导致一部分人无法享受到科技所带来的便利。因此,信息化平权显得尤为重要,它能够消除数字鸿沟,让每个人都能平等分享科技发展的成果,实现社会的全面进步

See all articles

Hot AI Tools

Undresser.AI Undress

Undresser.AI Undress

AI-powered app for creating realistic nude photos

AI Clothes Remover

AI Clothes Remover

Online AI tool for removing clothes from photos.

Undress AI Tool

Undress AI Tool

Undress images for free

Clothoff.io

Clothoff.io

AI clothes remover

AI Hentai Generator

AI Hentai Generator

Generate AI Hentai for free.

Hot Article

R.E.P.O. Energy Crystals Explained and What They Do (Yellow Crystal)
2 weeks agoBy尊渡假赌尊渡假赌尊渡假赌
Repo: How To Revive Teammates
1 months agoBy尊渡假赌尊渡假赌尊渡假赌
Hello Kitty Island Adventure: How To Get Giant Seeds
4 weeks agoBy尊渡假赌尊渡假赌尊渡假赌

Hot Tools

SAP NetWeaver Server Adapter for Eclipse

SAP NetWeaver Server Adapter for Eclipse

Integrate Eclipse with SAP NetWeaver application server.

EditPlus Chinese cracked version

EditPlus Chinese cracked version

Small size, syntax highlighting, does not support code prompt function

Dreamweaver Mac version

Dreamweaver Mac version

Visual web development tools

Notepad++7.3.1

Notepad++7.3.1

Easy-to-use and free code editor

VSCode Windows 64-bit Download

VSCode Windows 64-bit Download

A free and powerful IDE editor launched by Microsoft