search
HomeTechnology peripheralsAIGaussian-LIC: The first multi-sensor fusion 3DGS-SLAM system (Zhejiang University & TUM)

多重感測器與3DGS的結合

「基於3D空間中的取樣進行計算密集型SLAM任務所需的即時能力」的隱含表達式需要基於NeRF-based SLAM方法。 3DGS以其快速的渲染速度和優越的視覺品質而引人注目。作為一種明確且可以解釋的表達法,3DGS使得場景編輯變得簡單,有利於眾多下游任務的執行。

現有的基於輻射場的SLAM系統,主要在光照良好的小規模室內環境中進行測試,並使用順序RGB-D或RGB輸入獲得令人滿意的結果。當這些方法擴展到具有挑戰性的大規模不受控制的室外場景時將會遇到困難,例如具有挑戰性的光照條件、雜亂的背景和高度動態的自身運動。

使用Gaussian-LIC 透過融合來自多個感測器的數據,包括3D雷射雷達、IMU和相機(稱為ALIC),在不受控制的室外場景中

  • 首個基於雷射雷達慣性相機融合的3DGS-SLAM系統,實現了高度精確的姿態追蹤和逼真的線上地圖建構。
  • 利用順序著色光達點雲作為先驗,以線上方式重建高斯地圖。利用精心設計的一系列策略,逐步擴展高斯地圖並自適應地控制其密度,以實現高效率和準確性。
  • 在室內外場景中經過廣泛驗證,展示了逼真地圖製作的最新表現。在惡劣條件下表現出穩健性,包括高度動態運動、低照明和缺乏幾何結構和視覺紋理。

具體方法

如圖1所示是LIC-SLAM系統流程。

Gaussian-LIC: The first multi-sensor fusion 3DGS-SLAM system (Zhejiang University & TUM)

3DGS的表示

#將場景表示為各向異性的3D高斯集合,每個高斯的變異數:

Gaussian-LIC: The first multi-sensor fusion 3DGS-SLAM system (Zhejiang University & TUM)

將3D高斯投影到影像平面上,獲得2D高斯分佈:

Gaussian-LIC: The first multi-sensor fusion 3DGS-SLAM system (Zhejiang University & TUM)

投影獲得的2D高斯透過不透明度加權方程式影響像素:

Gaussian-LIC: The first multi-sensor fusion 3DGS-SLAM system (Zhejiang University & TUM)

渲染輪廓圖像以確定像素是否包含了足夠的來自高斯地圖的資訊:

Gaussian-LIC: The first multi-sensor fusion 3DGS-SLAM system (Zhejiang University & TUM)

LiDAR-IMU-Camera里程計

里程計採用了先前的工作Coco-LIC作為追蹤前端,一個連續時間緊密耦合的雷射雷達慣性攝影機測距系統。透過整合三種互補的感測器,Gaussian-LIC實現了強大、即時和高精度的姿態估計,從而促進了整個SLAM系統的運作。

每當最新的 (本文為0.1秒)內的雷射雷達慣性攝影機資料被累積時,會基於因子圖有效地優化該時間間隔內的軌跡:

  • 光達點雲到地圖
  • 影像到地圖的視覺因子
  • IMU因子
  • 偏差因子

在時間間隔內我們僅利用最新的影像進行優化。

Gaussian-LIC維護一個以0.1公尺解析度儲存在體素中的全域雷射雷達地圖。在優化當前時間間隔內的軌跡後,為了提高效率,僅將該時間段內四分之一的激光雷達點轉換為世界坐標並附加到體素中,以確保每個體素中的激光雷達點之間的距離應大於0.05米,以減少冗餘。對於成功附加的光達點,將它們投影到最新的圖像上,並透過查詢像素值為其著色。隨後,它們與估計的相機姿態的最新圖像一起發送到映射線程,被視為一幀。

基於3DGS的逼真建圖

隨著持續輸入:估計的相機姿態、無畸變影像、著色的雷射雷達點,建圖模組逐步建構和優化高斯地圖,主要分4步:

  1. 初始化:利用在第一個時間間隔內接收到的所有雷射光測點來創建新的高斯分佈。為了減輕混疊偽影,我們為距離較近的高斯分佈分配較小的尺度,而對於遠離影像平面的高斯分佈則分配較大的尺度:

Gaussian-LIC: The first multi-sensor fusion 3DGS-SLAM system (Zhejiang University & TUM)

  1. Extension: Normally, each frame received captures the geometry and appearance of a new viewing area. In order to avoid duplication or extremely similar information redundancy, first render a contour image from the current image view according to equation (6), and generate a mask to select the current Gaussian Unreliable pixels in the map and tend to observe new areas, only lidar points that can be projected onto the selected pixel will be exploited:

Gaussian-LIC: The first multi-sensor fusion 3DGS-SLAM system (Zhejiang University & TUM)

  1. Optimization: Optimization using all received images is computationally infeasible for our online incremental mapping system, so every fifth image frame is considered a keyframe. To keep the computational complexity bounded for efficiency, we select active keyframes to optimize the Gaussian map, including the first and last keyframe, and other randomly selected keyframes, avoiding the catastrophic forgetting problem and maintaining the geometric consistency of the global map. We randomly shuffle the selected keyframes and iterate through each keyframe one by one, optimizing the map by minimizing the re-rendering loss:

Gaussian-LIC: The first multi-sensor fusion 3DGS-SLAM system (Zhejiang University & TUM)

  1. Adaptive Control: LiDAR scanning may not cover the entire scene, especially in borderless outdoor environments. It is therefore necessary to perform adaptive control to densify the Gaussian map within unit volume. The original strategy handles Gaussians with large gradients at fixed intervals, while in incremental mode, since they often have not converged yet, recently added Gaussians may be incorrectly cloned or split due to having large gradients. The solution is to classify the existing Gaussian distributions into stable and unstable categories, and only densify the stable Gaussian distributions.

experiment

Gaussian-LIC: The first multi-sensor fusion 3DGS-SLAM system (Zhejiang University & TUM)

Gaussian-LIC: The first multi-sensor fusion 3DGS-SLAM system (Zhejiang University & TUM)

Gaussian-LIC: The first multi-sensor fusion 3DGS-SLAM system (Zhejiang University & TUM)

Gaussian-LIC: The first multi-sensor fusion 3DGS-SLAM system (Zhejiang University & TUM)

Gaussian-LIC: The first multi-sensor fusion 3DGS-SLAM system (Zhejiang University & TUM)

Summary and Discussion

Gaussian-LIC is a novel LiDAR-inertial-camera radiation field SLAM system, It achieves the dual functions of highly accurate tracking and realistic online reconstruction with 3D Gaussian splatter. Shaded LiDAR points in world coordinates produced by efficient LiDAR-inertial-camera odometry are used as good priors to assist in the incremental building of Gaussian maps with significantly better generalization in indoor and outdoor scenes .

Room for improvement:

  • Try to remove unnecessary Gaussian distributions in the map.
  • Supports various lidar, currently only trying to use solid-state lidar.
  • The camera's automatic exposure and motion blur affect the visual quality of realistic maps.

The above is the detailed content of Gaussian-LIC: The first multi-sensor fusion 3DGS-SLAM system (Zhejiang University & TUM). For more information, please follow other related articles on the PHP Chinese website!

Statement
This article is reproduced at:51CTO.COM. If there is any infringement, please contact admin@php.cn delete
Wow Awesome!三星 Galaxy Ring 体验:2999 元的真 · 智能戒指Wow Awesome!三星 Galaxy Ring 体验:2999 元的真 · 智能戒指Jul 19, 2024 pm 02:31 PM

三星在7月17日正式发布了国行版的三星GalaxyRing,定价2999元。GalaxyRing的真机,真就是2024版的"WowAwesome,这是我独享的moment"。它是近几年除苹果VisionPro之外,让我们感觉最新鲜的电子产品(虽然听起来像是在立flag)。(图中,左右两边的戒指,就是GalaxyRing↑)三星GalaxyRing规格(国行官网数据):ZephyrRTOS系统,8MB存储;10ATM防水+IP68;电池容量18mAh到23.5mAh(不同尺码的

升级全面屏!iPhone SE4 提前至 9 月升级全面屏!iPhone SE4 提前至 9 月Jul 24, 2024 pm 12:56 PM

最近微博上爆料了关于iPhoneSE4的新消息,消息称iPhoneSE4的后盖工艺和iPhone16标准版完全一样,也就是说,iPhoneSE4会采用玻璃背板,并搭配直屏直边的设计。消息称iPhoneSE4将会提前至今年9月发布,也就是很有可能会和iPhone16同时亮相。1.根据曝光的渲染图可见,iPhoneSE4正面设计与iPhone13相近,刘海屏上设置有前置摄像头和FaceID传感器。背面采用类似iPhoneXr的布局,但仅配置一颗摄像头,并未设有整体摄像模组。

手机1英寸传感器到底有多大 其实比相机的1英寸还大手机1英寸传感器到底有多大 其实比相机的1英寸还大May 08, 2024 pm 06:40 PM

昨天的文章里没提“传感器尺寸”,没想到大家原来有这么多误解……1英寸到底是多少?因为一些历史遗留问题*,无论是相机还是手机,传感器对角线长度中的“1英寸”都不是25.4mm。*涉及到真空管,这里不做展开,有点类似马屁股决定铁轨宽度。为了避免误读,更严谨的写法是“1.0型”或者“Type1.0”。并且,当传感器尺寸小于1/2型时,1型=18mm;而在传感器尺寸大于等于1/2型时,1型=

AirPods Pro 和 AirPods 3:如何更改力传感器控件AirPods Pro 和 AirPods 3:如何更改力传感器控件Apr 13, 2023 am 11:43 AM

AirPods Pro具有出色的功能,例如降噪和采用入耳式设计的透明模式。为了在这些功能和其他功能之间切换,AirPods Pro(和 AirPods 3)杆中的力传感器可提供轻松控制。继续阅读以了解如何更改 AirPods 控件,包括按压速度、按压持续时间以及左右力传感器的功能等等。AirPods gen 1 和 2使用耳塞上的双击进行控制。但AirPods Pro 和 AirPods 3使用内置在每个耳塞杆中的压力传感器。我们将介绍三种在降噪、透明度和关闭之间切换的方法,如何自定义力度传感器

为什么有人认为安卓拍照可以吊打苹果?答案就这么直接为什么有人认为安卓拍照可以吊打苹果?答案就这么直接Mar 25, 2024 am 09:50 AM

在讨论安卓手机的拍照功能时,大多数用户对其给予了肯定,与苹果手机相比,用户普遍认为安卓手机的拍照表现更出色。这一观点并非毫无根据,实际原因也是显而易见的。高端安卓手机在硬件配置方面具有较大的竞争优势,尤其是摄像头传感器方面。许多高端安卓手机采用最新的、顶级的摄像头传感器,这些传感器在像素数、光圈大小和光学变焦能力等方面往往比同期发布的iPhone更为突出。这种优势使得安卓手机在拍摄照片和录制视频时能够提供更高质量的成像效果,满足用户对于摄影和摄像的需求。因此,硬件配置的竞争优势成为了安卓手机吸引

国产传感器天花板!OV50K40传感器详解国产传感器天花板!OV50K40传感器详解Mar 20, 2024 am 09:56 AM

国产制造商豪威发布了新CMOS——OV50K40传感器。这款传感器是全球首个采用TheiaCel技术的智能手机图像传感器,其独特的设计使得单次曝光能够实现接近人眼级别的动态范围。OV50K40传感器拥有5000万像素的原生分辨率,并采用了1/1.3英寸的大底面积,每个像素的面积达到了1.2微米。此外,它还支持像素四合一输出,即将四个像素合并成一个,输出1250万像素的图像,从而带来更强大的感光能力。这款传感器的帧率支持120帧/秒和60帧/秒(HDR),可实现完整的5000万像素B

Gaussian-LIC:首个多传感器融合3DGS-SLAM系统(浙大&TUM)Gaussian-LIC:首个多传感器融合3DGS-SLAM系统(浙大&TUM)Apr 29, 2024 am 11:49 AM

多传感器与3DGS的结合“基于3D空间中的采样进行计算密集型SLAM任务所需的实时能力”的隐式表达法需要基于NeRF-based+SLAM方法。3DGS以其快速的渲染速度和优越的视觉质量而引人注目。作为一种明确且可以解释的表达法,3DGS使得场景编辑变得简单,有利于众多下游任务的执行。现有的基于辐射场的SLAM系统,主要在光照良好的小规模室内环境中进行测试,并使用顺序RGB-D或RGB输入获得令人满意的结果。当这些方法扩展到具有挑战性的大规模不受控制的室外场景时将会遇到困难,例如具有挑战性的光照

如何使用 HomePod 和 HomePod Mini 上的温度和湿度传感器如何使用 HomePod 和 HomePod Mini 上的温度和湿度传感器May 24, 2023 am 08:52 AM

2023年1月,Apple发布了HomePod16.3软件更新,为第二代‌HomePod‌和所有HomePodmini型号增加了湿度和温度感应功能。本文介绍了如何使用这些功能。2023年1月,Apple发布了第二代全尺寸HomePod,配备可用于测量室内气候的温度和湿度传感器,同时还发布了软件更新,为现有的HomePodmini型号带来了相同的功能。众所周知,Apple的HomePodmini具有未使用的温度和湿度传感器,但现在Apple更新了软件,这些传感器

See all articles

Hot AI Tools

Undresser.AI Undress

Undresser.AI Undress

AI-powered app for creating realistic nude photos

AI Clothes Remover

AI Clothes Remover

Online AI tool for removing clothes from photos.

Undress AI Tool

Undress AI Tool

Undress images for free

Clothoff.io

Clothoff.io

AI clothes remover

AI Hentai Generator

AI Hentai Generator

Generate AI Hentai for free.

Hot Article

R.E.P.O. Energy Crystals Explained and What They Do (Yellow Crystal)
3 weeks agoBy尊渡假赌尊渡假赌尊渡假赌
R.E.P.O. Best Graphic Settings
3 weeks agoBy尊渡假赌尊渡假赌尊渡假赌
R.E.P.O. How to Fix Audio if You Can't Hear Anyone
3 weeks agoBy尊渡假赌尊渡假赌尊渡假赌

Hot Tools

VSCode Windows 64-bit Download

VSCode Windows 64-bit Download

A free and powerful IDE editor launched by Microsoft

WebStorm Mac version

WebStorm Mac version

Useful JavaScript development tools

DVWA

DVWA

Damn Vulnerable Web App (DVWA) is a PHP/MySQL web application that is very vulnerable. Its main goals are to be an aid for security professionals to test their skills and tools in a legal environment, to help web developers better understand the process of securing web applications, and to help teachers/students teach/learn in a classroom environment Web application security. The goal of DVWA is to practice some of the most common web vulnerabilities through a simple and straightforward interface, with varying degrees of difficulty. Please note that this software

SecLists

SecLists

SecLists is the ultimate security tester's companion. It is a collection of various types of lists that are frequently used during security assessments, all in one place. SecLists helps make security testing more efficient and productive by conveniently providing all the lists a security tester might need. List types include usernames, passwords, URLs, fuzzing payloads, sensitive data patterns, web shells, and more. The tester can simply pull this repository onto a new test machine and he will have access to every type of list he needs.

Atom editor mac version download

Atom editor mac version download

The most popular open source editor