Multimodal Data Fusion for Perception Systems in Autonomous Driving

Abstract

 A large number of environmental sensors and various data types can be used for multiple purposes. This paper will propose a multi-level sensor fusion method that systematically integrates data from LiDAR, radar, and cameras to address the issue of unreliable perception in complex dynamic traffic environments. The three modules in this paper include sensor-specific encoding, precise spatiotemporal alignment, and adaptive attention-based fusion. A large number of experiments were conducted on a large-scale urban driving dataset with various lighting and weather conditions, as well as multiple occlusion scenarios. The proposed method achieved an average detection accuracy of 90.5%, surpassing the benchmarks of single-modal LiDAR (87.2%), radar (86.0%), and camera-only (82.4%). The fusion model remains stable in sensor failure and high-noise environments, achieving high F1 scores in the more challenging categories of cyclists and emergency vehicles. Experiments show that the system demonstrates good detection accuracy and completeness under complex operating conditions. The above results support the construction of a highly reliable and scalable autonomous driving vision system thru hierarchical multimodal fusion. This provides strong support for the continuous development of intelligent transportation technology.