How to closely integrate online mapping and trajectory prediction?

Original title: Producing and Leveraging Online Map Uncertainty in Trajectory Prediction

Paper link: https://arxiv.org/pdf/2403.16439.pdf

Code link: https ://github.com/alfredgu001324/MapUncertaintyPrediction

Author affiliation: Vector Institute NVIDIA Research, University of Toronto, Stanford University

##Thesis idea:

High-definition (HD) maps have played an integral role in the development of the modern autonomous vehicle (AV) technology stack, despite the high annotation and maintenance costs associated with this. Therefore, many recent works propose methods for online estimation of HD maps from sensor data, complicating integration in autonomous driving technology stacks. In particular, they do not generate uncertainty or confidence estimates. This paper extends multiple state-of-the-art online map estimation methods to enable additional estimates of uncertainty and improve predictive power by up to 15% on the real-world nuScenes driving dataset. In the process, we find that incorporating uncertainty improves training convergence by up to 50% and improves predictive power by up to 15% on the real-world nuScenes driving dataset.

Main contributions:

This paper introduces a general vectorized map uncertainty description and extends many state-of-the-art online map estimation methods to make them additional Output uncertainty estimates without degrading pure mapping performance.

This paper empirically analyzes the potential sources of map uncertainty, confirms the lack of confidence in current map estimation methods, and provides information for future research directions.

This article will discuss recent online map estimation models combined with multiple state-of-the-art trajectory prediction methods, and show how the performance and training characteristics of downstream prediction models can be significantly improved by incorporating online mapping uncertainty. Accelerate training convergence by up to 50% and improve online prediction accuracy by up to 15%.

Network Design:

A key component of autonomous driving is understanding the static environment, such as the road layout and traffic surrounding autonomous vehicles (AVs). Therefore, high-precision (HD) maps have been developed to capture and provide such information, containing semantic information such as road boundaries, lane dividers, and centimeter-level road markings. In recent years, HD maps have proven to be indispensable for the development and deployment of autonomous vehicles and are widely used today [35]. However, HD maps are expensive to annotate and maintain over time, and they can only be used in geofenced areas, which limits the scalability of autonomous vehicles. ”.

To address these issues, many recent studies turn to online estimation of high-precision (HD) maps from sensor data. Broadly speaking, they aim to predict the locations and categories of map elements, usually in the form of polygons or polylines form, all derived from camera images and lidar (LiDAR) scans. However, current online map estimation methods do not produce any relevant uncertainty or confidence information. This is problematic because it leads to downstream users (consumers) implicitly assume that the inferred map components are deterministic, and any mapping errors (e.g., incorrect movement or placement of map elements) may lead to erroneous downstream behavior. To this end, this paper proposes to reveal the differences in online map estimation methods. of map uncertainty and incorporate it into downstream modules. Specifically, this paper incorporates map uncertainty into trajectory prediction and finds that mapper-predictor systems that incorporate map uncertainty (Figure 1) are better than those that do not. Compared with the system with map uncertainty, the performance is significantly improved.

Figure 1. Uncertainty is generated from the online high-precision (HD) map estimation method and the Its incorporation into downstream modules brings various benefits. Left: Real HD map and agent location. Middle: HiVT[41] prediction using MapTR[22] output map. Right: Using MapTR[22] output The map and the HiVT[41] prediction with added point uncertainty (due to the large uncertainty due to the left road boundary being blocked by parked vehicles).

Fig. 2. Many online high-precision vector map estimation methods operate by encoding multi-camera images, transforming them into a common bird's-eye view (BEV) feature space, and regressing the vertices of map elements. Our work does this by adding a probabilistic regression head This common output structure is enhanced by modeling each map vertex as a Laplacian distribution. To evaluate the resulting downstream effects, this paper further extends the downstream prediction model to encode map uncertainty, enhancing graph neural Network (GNN) and Transformer-based map encoder.

Experimental results:

Figure 3. The uncertainty representation proposed in this article can capture the differences between the camera of the autonomous vehicle (AV) and the surrounding environment. Uncertainty due to occlusion between map elements. Left: Images from the front and front right cameras. Right: HD map generated by the online high-precision map model enhanced in this article. The ellipse represents the standard deviation of the distribution. Colors represent road boundaries, lane dividers, crosswalks, and lane centerlines.

# Figure 4. In a dense parking lot, many models fail to produce accurate maps. Left: rear and rear left camera images. Right: HD map generated by the online high-precision map model enhanced in this article. The ellipse shows the standard deviation of the distribution. Colors represent road boundaries, lane dividers, crosswalks, and lane centerlines.

#Summary:

This paper proposes a general vectorized map uncertainty formula and extends a variety of the latest online map estimation methods, including MapTR [22], MapTRv2 [23 ] and StreamMapNet [38], enabling them to additionally output uncertainty. We systematically analyze the resulting uncertainty and find that our approach captures many sources of uncertainty (occlusions, distance from the camera, time of day, and weather). Finally, this paper combines these online map estimation models with state-of-the-art trajectory prediction methods (DenseTNT [13] and HiVT [41]) and shows that incorporating online map uncertainty significantly improves the performance and training characteristics of the prediction models, respectively. Up to 15% and 50%. An exciting future research direction is to use these uncertainty outputs to measure the calibration of map models (similar to [16]). However, this task is complicated by the need for fuzzy point set matching, which is a challenging problem in itself.

Citation:

Gu X, Song G, Gilitschenski I, et al. Producing and Leveraging Online Map Uncertainty in Trajectory Prediction[J]. arXiv preprint arXiv:2403.16439 , 2024.

The above is the detailed content of How to closely integrate online mapping and trajectory prediction?. For more information, please follow other related articles on the PHP Chinese website!