search
HomeTechnology peripheralsAIGaussian-LIC: The first multi-sensor fusion 3DGS-SLAM system (Zhejiang University & TUM)

多重感測器與3DGS的結合

「基於3D空間中的取樣進行計算密集型SLAM任務所需的即時能力」的隱含表達式需要基於NeRF-based SLAM方法。 3DGS以其快速的渲染速度和優越的視覺品質而引人注目。作為一種明確且可以解釋的表達法,3DGS使得場景編輯變得簡單,有利於眾多下游任務的執行。

現有的基於輻射場的SLAM系統,主要在光照良好的小規模室內環境中進行測試,並使用順序RGB-D或RGB輸入獲得令人滿意的結果。當這些方法擴展到具有挑戰性的大規模不受控制的室外場景時將會遇到困難,例如具有挑戰性的光照條件、雜亂的背景和高度動態的自身運動。

使用Gaussian-LIC 透過融合來自多個感測器的數據,包括3D雷射雷達、IMU和相機(稱為ALIC),在不受控制的室外場景中

  • 首個基於雷射雷達慣性相機融合的3DGS-SLAM系統,實現了高度精確的姿態追蹤和逼真的線上地圖建構。
  • 利用順序著色光達點雲作為先驗,以線上方式重建高斯地圖。利用精心設計的一系列策略,逐步擴展高斯地圖並自適應地控制其密度,以實現高效率和準確性。
  • 在室內外場景中經過廣泛驗證,展示了逼真地圖製作的最新表現。在惡劣條件下表現出穩健性,包括高度動態運動、低照明和缺乏幾何結構和視覺紋理。

具體方法

如圖1所示是LIC-SLAM系統流程。

Gaussian-LIC: The first multi-sensor fusion 3DGS-SLAM system (Zhejiang University & TUM)

3DGS的表示

#將場景表示為各向異性的3D高斯集合,每個高斯的變異數:

Gaussian-LIC: The first multi-sensor fusion 3DGS-SLAM system (Zhejiang University & TUM)

將3D高斯投影到影像平面上,獲得2D高斯分佈:

Gaussian-LIC: The first multi-sensor fusion 3DGS-SLAM system (Zhejiang University & TUM)

投影獲得的2D高斯透過不透明度加權方程式影響像素:

Gaussian-LIC: The first multi-sensor fusion 3DGS-SLAM system (Zhejiang University & TUM)

渲染輪廓圖像以確定像素是否包含了足夠的來自高斯地圖的資訊:

Gaussian-LIC: The first multi-sensor fusion 3DGS-SLAM system (Zhejiang University & TUM)

LiDAR-IMU-Camera里程計

里程計採用了先前的工作Coco-LIC作為追蹤前端,一個連續時間緊密耦合的雷射雷達慣性攝影機測距系統。透過整合三種互補的感測器,Gaussian-LIC實現了強大、即時和高精度的姿態估計,從而促進了整個SLAM系統的運作。

每當最新的 (本文為0.1秒)內的雷射雷達慣性攝影機資料被累積時,會基於因子圖有效地優化該時間間隔內的軌跡:

  • 光達點雲到地圖
  • 影像到地圖的視覺因子
  • IMU因子
  • 偏差因子

在時間間隔內我們僅利用最新的影像進行優化。

Gaussian-LIC維護一個以0.1公尺解析度儲存在體素中的全域雷射雷達地圖。在優化當前時間間隔內的軌跡後,為了提高效率,僅將該時間段內四分之一的激光雷達點轉換為世界坐標並附加到體素中,以確保每個體素中的激光雷達點之間的距離應大於0.05米,以減少冗餘。對於成功附加的光達點,將它們投影到最新的圖像上,並透過查詢像素值為其著色。隨後,它們與估計的相機姿態的最新圖像一起發送到映射線程,被視為一幀。

基於3DGS的逼真建圖

隨著持續輸入:估計的相機姿態、無畸變影像、著色的雷射雷達點,建圖模組逐步建構和優化高斯地圖,主要分4步:

  1. 初始化:利用在第一個時間間隔內接收到的所有雷射光測點來創建新的高斯分佈。為了減輕混疊偽影,我們為距離較近的高斯分佈分配較小的尺度,而對於遠離影像平面的高斯分佈則分配較大的尺度:

Gaussian-LIC: The first multi-sensor fusion 3DGS-SLAM system (Zhejiang University & TUM)

  1. Extension: Normally, each frame received captures the geometry and appearance of a new viewing area. In order to avoid duplication or extremely similar information redundancy, first render a contour image from the current image view according to equation (6), and generate a mask to select the current Gaussian Unreliable pixels in the map and tend to observe new areas, only lidar points that can be projected onto the selected pixel will be exploited:

Gaussian-LIC: The first multi-sensor fusion 3DGS-SLAM system (Zhejiang University & TUM)

  1. Optimization: Optimization using all received images is computationally infeasible for our online incremental mapping system, so every fifth image frame is considered a keyframe. To keep the computational complexity bounded for efficiency, we select active keyframes to optimize the Gaussian map, including the first and last keyframe, and other randomly selected keyframes, avoiding the catastrophic forgetting problem and maintaining the geometric consistency of the global map. We randomly shuffle the selected keyframes and iterate through each keyframe one by one, optimizing the map by minimizing the re-rendering loss:

Gaussian-LIC: The first multi-sensor fusion 3DGS-SLAM system (Zhejiang University & TUM)

  1. Adaptive Control: LiDAR scanning may not cover the entire scene, especially in borderless outdoor environments. It is therefore necessary to perform adaptive control to densify the Gaussian map within unit volume. The original strategy handles Gaussians with large gradients at fixed intervals, while in incremental mode, since they often have not converged yet, recently added Gaussians may be incorrectly cloned or split due to having large gradients. The solution is to classify the existing Gaussian distributions into stable and unstable categories, and only densify the stable Gaussian distributions.

experiment

Gaussian-LIC: The first multi-sensor fusion 3DGS-SLAM system (Zhejiang University & TUM)

Gaussian-LIC: The first multi-sensor fusion 3DGS-SLAM system (Zhejiang University & TUM)

Gaussian-LIC: The first multi-sensor fusion 3DGS-SLAM system (Zhejiang University & TUM)

Gaussian-LIC: The first multi-sensor fusion 3DGS-SLAM system (Zhejiang University & TUM)

Gaussian-LIC: The first multi-sensor fusion 3DGS-SLAM system (Zhejiang University & TUM)

Summary and Discussion

Gaussian-LIC is a novel LiDAR-inertial-camera radiation field SLAM system, It achieves the dual functions of highly accurate tracking and realistic online reconstruction with 3D Gaussian splatter. Shaded LiDAR points in world coordinates produced by efficient LiDAR-inertial-camera odometry are used as good priors to assist in the incremental building of Gaussian maps with significantly better generalization in indoor and outdoor scenes .

Room for improvement:

  • Try to remove unnecessary Gaussian distributions in the map.
  • Supports various lidar, currently only trying to use solid-state lidar.
  • The camera's automatic exposure and motion blur affect the visual quality of realistic maps.

The above is the detailed content of Gaussian-LIC: The first multi-sensor fusion 3DGS-SLAM system (Zhejiang University & TUM). For more information, please follow other related articles on the PHP Chinese website!

Statement
This article is reproduced at:51CTO.COM. If there is any infringement, please contact admin@php.cn delete
The AI Skills Gap Is Slowing Down Supply ChainsThe AI Skills Gap Is Slowing Down Supply ChainsApr 26, 2025 am 11:13 AM

The term "AI-ready workforce" is frequently used, but what does it truly mean in the supply chain industry? According to Abe Eshkenazi, CEO of the Association for Supply Chain Management (ASCM), it signifies professionals capable of critic

How One Company Is Quietly Working To Transform AI ForeverHow One Company Is Quietly Working To Transform AI ForeverApr 26, 2025 am 11:12 AM

The decentralized AI revolution is quietly gaining momentum. This Friday in Austin, Texas, the Bittensor Endgame Summit marks a pivotal moment, transitioning decentralized AI (DeAI) from theory to practical application. Unlike the glitzy commercial

Nvidia Releases NeMo Microservices To Streamline AI Agent DevelopmentNvidia Releases NeMo Microservices To Streamline AI Agent DevelopmentApr 26, 2025 am 11:11 AM

Enterprise AI faces data integration challenges The application of enterprise AI faces a major challenge: building systems that can maintain accuracy and practicality by continuously learning business data. NeMo microservices solve this problem by creating what Nvidia describes as "data flywheel", allowing AI systems to remain relevant through continuous exposure to enterprise information and user interaction. This newly launched toolkit contains five key microservices: NeMo Customizer handles fine-tuning of large language models with higher training throughput. NeMo Evaluator provides simplified evaluation of AI models for custom benchmarks. NeMo Guardrails implements security controls to maintain compliance and appropriateness

AI Paints A New Picture For The Future Of Art And DesignAI Paints A New Picture For The Future Of Art And DesignApr 26, 2025 am 11:10 AM

AI: The Future of Art and Design Artificial intelligence (AI) is changing the field of art and design in unprecedented ways, and its impact is no longer limited to amateurs, but more profoundly affecting professionals. Artwork and design schemes generated by AI are rapidly replacing traditional material images and designers in many transactional design activities such as advertising, social media image generation and web design. However, professional artists and designers also find the practical value of AI. They use AI as an auxiliary tool to explore new aesthetic possibilities, blend different styles, and create novel visual effects. AI helps artists and designers automate repetitive tasks, propose different design elements and provide creative input. AI supports style transfer, which is to apply a style of image

How Zoom Is Revolutionizing Work With Agentic AI: From Meetings To MilestonesHow Zoom Is Revolutionizing Work With Agentic AI: From Meetings To MilestonesApr 26, 2025 am 11:09 AM

Zoom, initially known for its video conferencing platform, is leading a workplace revolution with its innovative use of agentic AI. A recent conversation with Zoom's CTO, XD Huang, revealed the company's ambitious vision. Defining Agentic AI Huang d

The Existential Threat To UniversitiesThe Existential Threat To UniversitiesApr 26, 2025 am 11:08 AM

Will AI revolutionize education? This question is prompting serious reflection among educators and stakeholders. The integration of AI into education presents both opportunities and challenges. As Matthew Lynch of The Tech Edvocate notes, universit

The Prototype: American Scientists Are Looking For Jobs AbroadThe Prototype: American Scientists Are Looking For Jobs AbroadApr 26, 2025 am 11:07 AM

The development of scientific research and technology in the United States may face challenges, perhaps due to budget cuts. According to Nature, the number of American scientists applying for overseas jobs increased by 32% from January to March 2025 compared with the same period in 2024. A previous poll showed that 75% of the researchers surveyed were considering searching for jobs in Europe and Canada. Hundreds of NIH and NSF grants have been terminated in the past few months, with NIH’s new grants down by about $2.3 billion this year, a drop of nearly one-third. The leaked budget proposal shows that the Trump administration is considering sharply cutting budgets for scientific institutions, with a possible reduction of up to 50%. The turmoil in the field of basic research has also affected one of the major advantages of the United States: attracting overseas talents. 35

All About Open AI's Latest GPT 4.1 Family - Analytics VidhyaAll About Open AI's Latest GPT 4.1 Family - Analytics VidhyaApr 26, 2025 am 10:19 AM

OpenAI unveils the powerful GPT-4.1 series: a family of three advanced language models designed for real-world applications. This significant leap forward offers faster response times, enhanced comprehension, and drastically reduced costs compared t

See all articles

Hot AI Tools

Undresser.AI Undress

Undresser.AI Undress

AI-powered app for creating realistic nude photos

AI Clothes Remover

AI Clothes Remover

Online AI tool for removing clothes from photos.

Undress AI Tool

Undress AI Tool

Undress images for free

Clothoff.io

Clothoff.io

AI clothes remover

Video Face Swap

Video Face Swap

Swap faces in any video effortlessly with our completely free AI face swap tool!

Hot Tools

WebStorm Mac version

WebStorm Mac version

Useful JavaScript development tools

mPDF

mPDF

mPDF is a PHP library that can generate PDF files from UTF-8 encoded HTML. The original author, Ian Back, wrote mPDF to output PDF files "on the fly" from his website and handle different languages. It is slower than original scripts like HTML2FPDF and produces larger files when using Unicode fonts, but supports CSS styles etc. and has a lot of enhancements. Supports almost all languages, including RTL (Arabic and Hebrew) and CJK (Chinese, Japanese and Korean). Supports nested block-level elements (such as P, DIV),

EditPlus Chinese cracked version

EditPlus Chinese cracked version

Small size, syntax highlighting, does not support code prompt function

DVWA

DVWA

Damn Vulnerable Web App (DVWA) is a PHP/MySQL web application that is very vulnerable. Its main goals are to be an aid for security professionals to test their skills and tools in a legal environment, to help web developers better understand the process of securing web applications, and to help teachers/students teach/learn in a classroom environment Web application security. The goal of DVWA is to practice some of the most common web vulnerabilities through a simple and straightforward interface, with varying degrees of difficulty. Please note that this software

SublimeText3 English version

SublimeText3 English version

Recommended: Win version, supports code prompts!