Please wait a minute...
 首页  期刊介绍 期刊订阅 联系我们 横山亮次奖 百年刊庆
 
最新录用  |  预出版  |  当期目录  |  过刊浏览  |  阅读排行  |  下载排行  |  引用排行  |  横山亮次奖  |  百年刊庆
清华大学学报(自然科学版)  2018, Vol. 58 Issue (2): 137-142    DOI: 10.16511/j.cnki.qhdxxb.2018.26.005
  计算机科学与技术 本期目录 | 过刊浏览 | 高级检索 |
基于K-means聚类特征消减的网络异常检测
贾凡1, 严妍2, 张家琪1
1. 北京交通大学 通信与信息系统北京市重点实验室, 北京 100044;
2. 中国信息安全认证中心, 北京 100020
K-means based feature reduction for network anomaly detection
JIA Fan1, YAN Yan2, ZHANG Jiaqi1
1. Key Laboratory of Communication & Information Systems of Beijing, Beijing Jiaotong University, Beijing 100044, China;
2. China Information Security Certification Center, Beijing 100020, China
全文: PDF(1241 KB)  
输出: BibTeX | EndNote (RIS)      
摘要 针对基础K-means算法在KDD 99数据集中检测罕见攻击效果差且效率低下等问题,该文通过数据统计的方式对数据集中各维度与每类攻击类型的相关分析发现,罕见攻击极易被大量的常见攻击所淹没,而当常见攻击被移去时,这些威胁性更大的罕见攻击则能够被更好地识别出来。基于此,该文提出一种改进的基于K-means分层迭代的检测算法,通过有针对性的特征选择来降低K-means聚类的数据维度,经过多次属性消减的K-means聚类迭代操作可以更加精准地检测到不同异常类型的攻击。在KDD 99数据集上的实验结果表明:该算法对原基础的K-means检测算法难以检测到的罕见攻击类型U2R/R2L攻击检测率几乎达到99%左右。同时随着每次分层迭代聚类维度近50%的降低,进一步节省了约90%的异常检测时间。
服务
把本文推荐给朋友
加入引用管理器
E-mail Alert
RSS
作者相关文章
贾凡
严妍
张家琪
关键词 异常检测K-means特征消减U2RR2L    
Abstract:Although the basic K-means test was used for anomaly detection in the KDD 99 attack dataset, its accuracy and efficiency for detecting rare attacks needs to be improved. Rare attacks, which are usually greater threats, are easily hidden by common threats so the rare attacks can be more easily identified by removing common attacks. An improved hierarchical iterative K-means method was developed based on this finding to detect all kinds of anomalies using feature reduction through correlations to decrease classification the dimensions. The algorithm is able to detect almost every rare attack with a 99% succesful classification rate and for nearly real-time detection with 90% less computations on the KDD 99 data compared with the basic K-means algorithm.
Key wordsanomaly detection    K-means    feature reduction    U2R    R2L
收稿日期: 2017-05-31      出版日期: 2018-02-15
ZTFLH:  O242.21  
引用本文:   
贾凡, 严妍, 张家琪. 基于K-means聚类特征消减的网络异常检测[J]. 清华大学学报(自然科学版), 2018, 58(2): 137-142.
JIA Fan, YAN Yan, ZHANG Jiaqi. K-means based feature reduction for network anomaly detection. Journal of Tsinghua University(Science and Technology), 2018, 58(2): 137-142.
链接本文:  
http://jst.tsinghuajournals.com/CN/10.16511/j.cnki.qhdxxb.2018.26.005  或          http://jst.tsinghuajournals.com/CN/Y2018/V58/I2/137
  图1 KDD9 9数据集各类攻击记录数量分布
  表1 KDD9 9数据集属性分类
  图3 第1 1—2 0维度各类型数据异常值所占比例
  图4 第2 1—3 0维度各类型数据异常值所占比例
  图5 第3 1—4 1维度各类型数据异常值所占比例
  表2 基础K G m e a n s算法与改进算法的比较
[1] NI X J, HE D J, FAROOQ A. Practical network anomaly detection using data mining techniques[J]. VFAST Transactions on Software Engineering, 2016, 9(2):1-6.
[2] TROST R. Practical intrusion analysis:Prevention and detection for the twenty-first century[M]. New York:Addison-Wesley, 2009.
[3] BHUYAN M H, BHATTACHARYYA D K, KALITA J K. Network anomaly detection:Methods, systems and tools[J]. IEEE Communications Surveys & Tutorials, 2014, 16(1):303-336.
[4] KNORR E M, NG R T. Algorithms for mining distance-based outliers in large datasets[C]//Proceedings of the 24th International Conference on Very Large Data Bases. New York, USA:Morgan Kaufmann, 1998:392-403.
[5] WEI L, QIAN W N, ZHOU A Y, et al. Hot:Hypergraph-based outlier test for categorical data[C]//Proceedings of the 7th Pacific-Asia Conference on Advances in Knowledge Discovery and Data Mining. Seoul, Korea:Springer, 2003:399-410.
[6] BAY S D, SCHWABACHER M. Mining distance-based outliers in near linear time with randomization and a simple pruning rule[C]//Proceedings of the 9th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. Washington, DC, USA:ACM Press, 2003:29-38.
[7] BREUNIG M M, KRIEGEL H P, NG R T, et al. LOF:Identifying density-based local outliers[J]. ACM SIGMOD Record, 2000, 29(2):93-104.
[8] 季成, 李晓东, 袁坚, 等. 基于<em>K</em>-means算法的DNS查询模式分析[J]. 清华大学学报(自然科学版), 2010, 50(4):601-604.JI C, LI X D, YUAN J, et al. Analysis of domain name queries based on the <em>K</em>-means algorithm[J]. Journal of Tsinghua University (Science and Technology), 2010, 50(4):601-604. (in Chinese)
[9] KDD Cup 1999 Intrusion detection dataset[EB/OL]. (1999-10-28). http://kdd.ics.uci.edu/databases/kddcup99/kddcup99.html.
[10] 蒋学英, 李雅珍, 严结苟. 基于SOM神经网络的异常检测算法研究[J]. 计算机科学, 2008, 35(10B):244-246. JIANG X Y, LI Y Z, YAN J G. Research on anomaly detection algorithm based on SOM neural network[J]. Computer Science, 2008, 35(10B):244-246. (in Chinese)
[11] MOUSTAFA N, SLAY J. The evaluation of network anomaly detection systems:Statistical analysis of the UNSW-NB15 data set and the comparison with the KDD 99 data set[J]. Information Security Journal:A Global Perspective, 2016, 25(1-3):18-31.
[12] WELLER-FAHY D J, BORGHETTI B J, SODEMANN A A. A survey of distance and similarity measures used within network intrusion anomaly detection[J]. IEEE Communications Surveys & Tutorials, 2014, 17(1):70-91.
[13] 傅涛, 孙文静, 孙亚民. 基于分箱统计的FCM算法及其在网络入侵检测中的应用[J]. 计算机科学, 2008, 35(4):36-39.FU T, SUN W J, SUN Y M. FCM algorithm based on Box-FCM statistics and its application in network intrusion detection[J]. Computer Science, 2008, 35(4):36-39. (in Chinese)
[14] SYARIF I, PRUGEL-BENNETT A, WILLS G. Unsupervised clustering approach for network anomaly detection[C]//International Conference on Networked Digital Technologies (NDT 2012). Berlin, Germany:Springer, 2012:135-145.
[1] 高洋, 任望, 吴润浦, 王卫苹, 伊胜伟, 韩白静. 信息物理系统的攻击检测与安全状态估计[J]. 清华大学学报(自然科学版), 2021, 61(11): 1234-1239.
[2] 王志国, 章毓晋. 监控视频异常检测:综述[J]. 清华大学学报(自然科学版), 2020, 60(6): 518-529.
[3] 梁杰, 陈嘉豪, 张雪芹, 周悦, 林家骏. 基于独热编码和卷积神经网络的异常检测[J]. 清华大学学报(自然科学版), 2019, 59(7): 523-529.
[4] 宋宇波, 杨慧文, 武威, 胡爱群, 高尚. 软件定义网络DDoS联合检测系统[J]. 清华大学学报(自然科学版), 2019, 59(1): 28-35.
[5] 陈兴蜀, 陈佳昕, 赵丹丹, 金鑫. 基于虚拟机IO序列与Markov模型的异常行为检测[J]. 清华大学学报(自然科学版), 2018, 58(4): 395-401,410.
[6] 彭勇, 向憧, 张淼, 陈冬青, 高海辉, 谢丰, 戴忠华. 工业控制系统场景指纹及异常检测[J]. 清华大学学报(自然科学版), 2016, 56(1): 14-21.
[7] 陈元琳, 柴跃廷, 刘义, 徐扬. 基于群体偏好的交易评价可信度[J]. 清华大学学报(自然科学版), 2015, 55(5): 558-564,571.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
版权所有 © 《清华大学学报(自然科学版)》编辑部
本系统由北京玛格泰克科技发展有限公司设计开发 技术支持:support@magtech.com.cn