A Novel Deep Learning Approach for Multi-Sensor Fusion in Autonomous Vehicle Perception

Abstract

An effective autonomous driving perception system must function well in real-world settings. In order to enhance situational robustness and environmental perception, LiDAR, radar, and camera data must be combined. This study examines the ongoing challenges in multi-sensor fusion. First, create a deep learning-based fusion framework that can manage the diverse spaces, timings, and semantics of the multiple sensors in a systematic manner. In order to implement attention-based fusion, adaptively extract features, and dynamically estimate uncertainty in the perception pipeline for context-aware decision-making, a new structure has been developed. Perform multi-stage attention weighting and cross-modal integration after methodically encoding and aligning each sensor stream separately. Experiments using a large public dataset have demonstrated that the suggested approach is more suited for real-world autonomous driving scenarios. The new framework is still a real-time system with minimal latency and an inference speed of 27 frames per second; quantitatively, it has improved the mean Average Precision (mAP) by more than 6 percentage points. To make sure that multiple item tracking and detection remain accurate, robustness has been evaluated in inclement weather and sensor deterioration. In summary, this study has offered a comprehensive and workable solution to the perception issue in intelligent vehicles, and experimental results have demonstrated its effectiveness, flexibility, and potential use in urban traffic scenarios.