Multi-Sensor Fusion of Thermal and Visible Cameras for UAV Navigation Target Recognition Applications

Abstract

Computer-based multi-sensor fusion can enhance the perception capabilities of drones (UAVs) in a new way, enabling them to tackle complex navigation and target recognition problems in the real world. This paper proposes a modular drone platform that combines visible spectrum RGB and thermal infrared cameras to mitigate the limitations of single-modal systems under adverse weather, fluctuating lighting conditions, and complex backgrounds. The system uses precise hardware synchronization, stable spatial calibration, and a multi-level fusion structure that combines pixel-level and feature-level data. For experimental evaluation, over 218,000 different synchronized image pairs were collected under bright, dim, dark, and low visibility conditions. According to the benchmark data, the proposed fusion method achieved an average accuracy of 87.6% under foggy and rainy conditions, which is 13% higher than the single-sensor baseline. The recall rate is also higher; in other words, the ratio of true positives to false negatives is relatively high, and the fusion system performs better in the challenge. In order to achieve real-time operation of drones, the detection delay should not exceed 90 milliseconds. According to comparisons of different scales, urban and rural areas are relatively common. The above results indicate that in order to achieve reliable and high-confidence identification under complex aerial monitoring conditions, multiple sensors need to be used. These advancements have made drones more practical in fields such as security, environmental monitoring, and industrial inspection.