Multi-Head Self-Attention Model for Anomaly Detection in Encrypted Network Traffic

Abstract

With the continuous development of encryption technology, it has become increasingly difficult to detect abnormal behavior in encrypted network traffic within current cybersecurity. To address the issue of detecting anomalies in encrypted traffic, this paper designs a multi-head self-attention neural network. A method using deep attention mechanisms and advanced feature engineering to model the complex feature dependencies in encrypted traffic streams, in order to more accurately distinguish between normal and abnormal behaviors. Extensive testing was conducted on a large-scale real-world encrypted traffic dataset with multiple protocols and operating environments. The new model achieved an accuracy of 98.6%, an F1-score of 97.7%, and a ROC-AUC of 0.996, indicating significant improvements over previous methods and those based on deep learning. The diversity of attention heads, feature selection, and composite loss design are crucial for the overall stability and detection performance of the system. Due to its good generalization ability and low-latency inference, this model can be used in high-throughput, dynamic network environments. Using multi-head self-attention and custom features to build a robust and scalable system for identifying anomalies in encrypted traffic lays a solid foundation for further research and applications in network security.