Unsupervised network traffic anomaly detection based on score iterations

PING Guolou; ZENG Tingyu; YE Xiaojun

doi:10.16511/j.cnki.qhdxxb.2021.21.045

PDF(3264 KB)

Journal of Tsinghua University(Science and Technology) ›› 2022, Vol. 62 ›› Issue (5) : 819-824. DOI: 10.16511/j.cnki.qhdxxb.2021.21.045

SPECIAL SECTION: VULNERABILITY ANALYSIS AND RISKA SSESSMENT

Unsupervised network traffic anomaly detection based on score iterations

{{article.zuoZhe_EN}}

Author information +

History +

Abstract

Network traffic anomaly detection is limited by the lack of annotation information in the traffic. This paper presents an unsupervised anomaly detection method based on score iterations that overcomes this limitation. An autoencoder based anomaly score iteration process was designed to learn generic anomaly features to determine an initial anomaly score. A deep ordinal regression model based anomaly score iteration process was then designed to learn discriminative anomaly features to further improve the anomaly score accuracy. Deep models, multi-view features and ensemble learning are also used to improve the detection accuracy. Tests on several datasets show that this method has significant advantages over other methods in the absence of annotation information and can be effectively applied to network traffic anomaly detection.

Key words

computer networks / anomaly scores / unsupervised / autoencoder / deep ordinal regression model / ensemble learning

Cite this article

EndNote

Ris (Procite)

Bibtex

Download Citations

PING Guolou, ZENG Tingyu, YE Xiaojun. Unsupervised network traffic anomaly detection based on score iterations[J]. Journal of Tsinghua University(Science and Technology). 2022, 62(5): 819-824 https://doi.org/10.16511/j.cnki.qhdxxb.2021.21.045

References

[1] LIU F T, TING K M, ZHOU Z H. Isolation forest[C]//Proceedings of the 2008 8th IEEE International Conference on Data Mining. Pisa, Italy:IEEE Press, 2008:413-422.
[2] ZHANG J, JONES K, SONG T Y, et al. Comparing unsupervised learning approaches to detect network intrusion using netflow data[C]//Proceedings of the 2017 Systems and Information Engineering Design Symposium. Charlottesville, USA:IEEE Press, 2017:122-127.
[3] ESKIN E, ARNOLD A, PRERAU M, et al. A geometric framework for unsupervised anomaly detection[M]//BARBARá D, JAJODIA S. Applications of Data Mining in Computer Security. Boston, MA, USA:Springer, 2002:77-101.
[4] RINGBERG H, SOULE A, REXFORD J, et al. Sensitivity of PCA for traffic anomaly detection[C]//Proceedings of the 2007 ACM SIGMETRICS International Conference on Measurement and Modeling of Computer Systems. California, USA:Association for Computing Machinery, 2007:109-120.
[5] PASCOAL C, DE OLIVEIRA M R, VALADAS R, et al. Robust feature selection and robust PCA for internet traffic anomaly detection[C]//2012 Proceedings IEEE INFOCOM. Orlando, USA:IEEE Press, 2012:1755-1763.
[6] MIRZA A H, COSAN S. Computer network intrusion detection using sequential LSTM neural networks autoencoders[C]//Proceedings of the 2018 26th Signal Processing and Communications Applications Conference. Izmir, Turkey:IEEE Press, 2018:1-4.
[7] MVNZ G, LI S, CARLE G. Traffic anomaly detection using k-means clustering[C]//Proceedings of Leistungs-, Zuverlässigkeits-und Verlässlichkeitsbewertung von Kommunikationsnetzen und Verteilten Systemen, 4 GI/ITG Workshop MMBnet. Hamburg, Germany, 2007:13-14.
[8] BOHARA A, THAKORE U, SANDERS W H. Intrusion detection in enterprise systems by combining and clustering diverse monitor data[C]//Proceedings of the Symposium and Bootcamp on the Science of Security. Pittsburgh, PA, USA:Association for Computing Machinery, 2016:7-16.
[9] VINCENT P, LAROCHELLE H, LAJOIE I, et al. Stacked denoising autoencoders:Learning useful representations in a deep network with a local denoising criterion[J]. Journal of Machine Learning Research, 2010, 11:3371-3408.
[10] MANDIC D P, CHAMBERS J. Recurrent neural networks for prediction:Learning algorithms, architectures and stability[M]. New York:John Wiley & Sons, Inc., 2001.
[11] LI Z, ZHAO Y, BOTTA N, et al. COPOD:Copula-based outlier detection[C]//Proceedings of the 2020 IEEE International Conference on Data Mining. Sorrento, Italy:IEEE Press, 2020:1118-1123.
[12] SHARAFALDIN I, LASHKARI A H, GHORBANI A A. Toward generating a new intrusion detection dataset and intrusion traffic characterization[C]//Proceedings of the 4th International Conference on Information Systems Security and Privacy. Funchal, Madeira Island, Portugal:SciTePress, 2018:108-116.
[13] MONTAZERISHATOORI M, DAVIDSON L, KAUR G, et al. Detection of DoH tunnels using time-series classification of encrypted traffic[C]//Proceedings of the 2020 IEEEInternational Conference on Dependable, Autonomic and Secure Computing, International Conference on Pervasive Intelligence and Computing, International Conference on Cloud and Big Data Computing, International Conference on Cyber Science and Technology Congress. Calgary, Canada:IEEE, 2020:63-70.
[14] RUFF L, VANDERMEULEN R A, GÖRNITZ N, et al. Deep one-class classification[C]//Proceedings of the 35th International Conference on Machine Learning. Stockholm, Sweden:PMLR, 2018:4393-4402.
[15] AYTEKIN C, NI X Y, CRICRI F, et al. Clustering and unsupervised anomaly detection with l2 normalized deep auto-encoder representations[C]//Proceedings of the 2018 International Joint Conference on Neural Networks. Rio de Janeiro, Brazil:IEEE, 2018:1-6.
[16] ZHAO Y, NASRULLAH Z, LI Z. PyOD:A Python toolbox for scalable outlier detection[J]. Journal of Machine Learning Research, 2019, 20:1-7.