Multi-Sensor Fusion Algorithms for Autonomous Vehicles

Abstract

By using sensor fusion and deep learning technologies, the perception capabilities of autonomous driving systems are advancing. This paper proposes an all-terrain, all-weather, single-sensor fusion framework. The four components of the system are cameras, LiDAR, radar, and inertial/ultrasonic sensors. The neural network architecture precisely integrates the extracted features in both time and space through attention-based fusion. A large number of experiments on distributed high-performance computing systems were conducted on the KITTI and nuScenes datasets. According to the above experiments, the proposed hybrid fusion model outperforms early fusion and single-modal baseline methods, achieving an accuracy of 0.91 during the day and an accuracy of 0.80 at night or under adverse weather conditions. The range of the median and the standard deviation of the results are both very small under different weather conditions. Based on the ablation experiments, each sensor has its own advantages. In terms of system efficiency, they can reach up to 27 frames per second on the RTX 3090 graphics card, and all have stable power consumption and memory. In summary, the aforementioned structure strikes an appropriate balance between detection performance, computational efficiency, and robustness to sensor degradation. This lays the foundation for the application of autonomous driving in real life.