Data-efficient 3D scene understanding for autonomous vehicles

Paper Title:

Multi-Modal Data-Efficient 3D Scene Understanding for Autonomous Driving

Published on:

8 May 2024

Primary Category:

Computer Vision and Pattern Recognition

Paper Authors:

Lingdong Kong,

Xiang Xu,

Jiawei Ren,

Wenwei Zhang,

Liang Pan,

Kai Chen,

Wei Tsang Ooi,

Ziwei Liu

Bullets

Key Details

•

Integrates LiDAR and camera data without needing extra image annotations

•

Manipulates laser beams between scans to exploit spatial priors

•

Distills semantic features from images to LiDAR point clouds

•

Generates auxiliary labels using CLIP for unlabeled data

•

Achieves high accuracy with 5x fewer labels than supervised methods

Explore the topics in this paper

3d scene understanding

autonomous driving

camera images

lidar point clouds

semi-supervised learning

AI generated summary

Data-efficient 3D scene understanding for autonomous vehicles

This paper proposes a semi-supervised framework called LaserMix++ that leverages both LiDAR point clouds and camera images to improve 3D scene understanding for autonomous driving with far less labeled data. Key innovations include multi-modal data mixing, transferring knowledge from images to point clouds, and generating auxiliary labels from language models, which enhance regularization and feature learning.