EVALUATION OF THE PERFORMANCE OF A MULTI-LEVEL MODEL FOR ANOMALOUS DNS QUERY DETECTION

Andrii Senyk

Abstract


Background. In modern network security systems, DNS (Domain Name System) traffic has become an increasingly attractive vector for covert data exfiltration and command-and-control communication. Existing machine learning methods frequently suffer from limited adaptability to novel attack patterns and an imbalance between detection accuracy and false positive rates.

Materials and Methods. This study proposes TunnelEye, a multi-level detection method for malicious DNS queries that integrates statistical feature analysis, structural n-gram modeling, and anomaly detection. Statistical properties of domain names, including string length, entropy, and alphanumeric ratio, are used for initial discrimination between benign and suspicious queries. Structural analysis based on character n-grams enables the identification of local patterns associated with encoded data such as Base32 and Base64. An autoencoder trained exclusively on legitimate DNS queries is employed as an independent anomaly detector to identify previously unseen and zero-day attacks. The supervised TunnelEye classifier and the autoencoder operate in parallel, each using an independently optimized F1-score based threshold to determine anomalous DNS queries.

Results and Discussion. Experimental evaluation using standard machine learning metrics (precision, recall, F1-score, ROC-AUC, PR-AUC, and false positive rate) demonstrates that TunnelEye consistently outperforms baseline statistical models and standalone autoencoders. The proposed method achieves high precision and recall while maintaining a minimal false positive rate. Experimental results show that TunnelEye achieves an average precision, recall, and F1-score of approximately 0.99, outperforming the baseline statistical model by more than 10% and significantly reducing the false positive rate.

Conclusion. TunnelEye provides a comprehensive and adaptive solution for malicious DNS query detection by combining supervised and unsupervised learning with dynamic threshold optimization. Its ability to balance detection accuracy and false positive reduction makes it well suited for deployment in modern enterprise cybersecurity systems for real-time DNS traffic monitoring.

Keywords: DNS traffic, anomaly detection, machine learning, multi-level model.


Full Text:

PDF

References


[1] Gonzalez Casanova, L. F., & Lin, P. C. (2021). Generalized classification of DNS over https traffic with deep learning. In 2021 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC), Tokyo, Japan, 2021, pp. 1903–1907. https://ieeexplore.ieee.org/document/9689667

[2] Ichise, H., Jin, Y., & Iida, K. (2023). Policy-based detection and blocking system against abnormal applications by analyzing DNS traffic. In 2023 22nd International Symposium on Communications and Information Technologies (ISCIT), Sydney, Australia, 2023, pp. 1–6. https://doi.org/10.1109/ISCIT57293.2023.10376042

[3] Zhang, C., Hu, X., Pan, X., Cheng, G., Li, R., & Wu, H. (2025). Accurate and early detection of IoT malware via DNS traffic analysis with deep learning. In ICC 2025 – IEEE International Conference on Communications, Montreal, QC, Canada, 2025, pp. 2665–2670. https://doi.org/10.1109/ICC52391.2025.11161323

[4] Ganesh, N., Parihar, A. S., & Ghosh, G. (2023). Analysing network traffic and implementing diverse technologies to examine different components of the network. In 2023 IEEE International Conference on ICT in Business Industry & Government (ICTBIG), Indore, India, 2023, pp. 1–10. https://doi.org/10.1109/ICTBIG59752.2023.10456258

[5] Harishkumar, S., & Bhuvaneswaran, R. S. (2024). Unveiling domain generation algorithms in DNS log traffic: A next-generation intelligent framework for dynamic anomaly detection and mitigation through machine learning analysis. In 2024 15th International Conference on Computing Communication and Networking Technologies (ICCCNT), Kamand, India, 2024, pp. 1–7. https://doi.org/10.1109/ICCCNT61001.2024.10726248

[6] Wu, X., Wang, X., Song, Y., & Ding, P. (2024). SSPT: A self supervised network traffic anomaly detection method. In 2024 20th International Conference on Mobility, Sensing and Networking (MSN), Harbin, China, 2024, pp. 1206–1207. https://doi.org/10.1109/MSN63567.2024.00177

[7] Zou, F., Ren, Y., Zhu, J., & Tang, J. (2021). Detecting data leakage in DNS traffic based on time series anomaly detection. In 2021 IEEE 23rd International Conference on High Performance Computing & Communications; 7th International Conference on Data Science & Systems; 19th International Conference on Smart City; 7th International Conference on Dependability in Sensor, Cloud & Big Data Systems & Application (HPCC/DSS/SmartCity/DependSys), Haikou, Hainan, China, 2021, pp. 503–510. https://doi.org/10.1109/HPCC-DSS-SmartCity-DependSys53884.2021.00090

[8] Du, X., et al. (2022). Design of an autoencoder-based anomaly detection for the DoH traffic system. In 2022 IEEE 25th International Conference on Computer Supported Cooperative Work in Design (CSCWD), Hangzhou, China, 2022, pp. 763–768. https://doi.org/10.1109/CSCWD54268.2022.9776029

[9] Hzami, M., Mahersia, H., & Bejaoui, T. (2025). Multi-level cyberbullying detection on social media using machine and deep learning models. In 2025 5th IEEE Middle East and North Africa Communications Conference (MENACOMM), Byblos, Lebanon, 2025, pp. 1–6. https://doi.org/10.1109/MENACOMM62946.2025.10911024

[10] Huang, X., Zhu, X., Xu, X., Zhu, M., Nian, A., & Guo, Y. (2022). Multi-granularity perceptual ensemble learning model with an application. In 2022 International Conference on Machine Learning, Cloud Computing and Intelligent Mining (MLCCIM), Xiamen, China, 2022, pp. 234–242. https://doi.org/10.1109/MLCCIM55934.2022.00047

[11] Wang, B., Xiong, G., Gou, G., Song, J., Li, Z., & Yang, Q. (2023). Identifying DoH tunnel traffic using core features and machine learning method. In 2023 26th International Conference on Computer Supported Cooperative Work in Design (CSCWD), Rio de Janeiro, Brazil, 2023, pp. 814–819. https://doi.org/10.1109/CSCWD57460.2023.10152678

[12] Rana, S., & Aksoy, A. (2021). Automated fast-flux detection using machine learning and genetic algorithms. In IEEE INFOCOM 2021 – IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS), Vancouver, BC, Canada, 2021, pp. 1–6. https://doi.org/10.1109/INFOCOMWKSHPS51825.2021.9484614




DOI: http://dx.doi.org/10.30970/eli.33.5

Refbacks

  • There are currently no refbacks.