Semantic Guided Pseudo-LiDAR Undistortion and Differentiated Sampling

• 1、School of Computer Science (National Pilot Software Engineering School), Beijing University of Posts and Telecommunications, Beijing, 100876

Abstract：Pseudo-LiDAR point cloud is a data format derived from depth maps and some parameters, resembling LiDAR point cloud.Through depth estimation and projection transformation,the raw data can be converted into Pseudo-LiDAR point cloud, which can improve the accuracy of localization and recognition in detection, and is also conducive to the fusion of multi-source heterogeneous data, demonstrating great potential in low-cost and highly flexible autonomous driving scenarios. Due to depth estimation errors and insufficient spatial information, there are still significant differences between pseudo-LiDAR and real LiDAR, which limits the improvement of detection accuracy. This paper proposes the following improvements to the existing pseudo-LiDAR generation and detection framework. We updated the stereo depth estimation network and 3D object detection network, and overcome domain gap through multi dataset training; in the case of projection parameter errors or missing information, semantic information and prior knowledge were used to fit the required parameters and correct the shape distortion of the pseudo-LiDAR; we improved pseudo-LiDAR confidence and sparsify the point clouds based on the error distribution pattern of depth estimation, thereby enhancing the accuracy of the pseudo-LiDAR in reflecting targets in the scene. Compared with current standard scheme, our optimized pseudo-LiDAR generation and detection framework achieved an average IoU(Intersection over Union) and AP(Average Precision) improvement of 15.23% and 14.03% respectively on the KITTI object detection dataset.

Keywords： 3D object detection Pseudo-LiDAR Depth estimation