Anomaly detection based on IO sequences in a virtual machine with the Markov mode

CHEN Xingshu; CHEN Jiaxin; ZHAO Dandan; JIN Xin

doi:10.16511/j.cnki.qhdxxb.2018.25.018

PDF(1324 KB)

Journal of Tsinghua University(Science and Technology) ›› 2018, Vol. 58 ›› Issue (4) : 395-401,410. DOI: 10.16511/j.cnki.qhdxxb.2018.25.018

COMPUTER SCIENCE AND TECHNOLOGY

Anomaly detection based on IO sequences in a virtual machine with the Markov mode

{{article.zuoZhe_EN}}

Author information +

History +

Abstract

A abnormal IO behavior in virtual machines is monitored to discover known and unknown virtual machine escape attacks. Hardware-assisted virtualization is used here in an anomaly detection method for IO sequences in virtual machines including asynchronous acquisition to efficiently collect the IO sequences of the virtual machine, relating the IO sequences with the processes running in the virtual machine for a fine-grained description of the virtual machine's IO behavior, and an algorithm for generating short IO sequences in virtual machines based on a double-layer hash table and a Markov chain model to detect the IO sequences of malicious virtual machines. A virtual machine detection system was implemented on a Kernel-based virtual machine (KVM) to evaluate the effectiveness of this system. The results show that the system can effectively detect some IO based on security threats and some known and unknown virtual machine escape attacks with an acceptable false alarm rate and performance overhead.

Key words

IO sequence / anomaly detection / virtual machine (VM) escape / Markov chain / short sequence

Cite this article

EndNote

Ris (Procite)

Bibtex

Download Citations

CHEN Xingshu, CHEN Jiaxin, ZHAO Dandan, JIN Xin. Anomaly detection based on IO sequences in a virtual machine with the Markov mode[J]. Journal of Tsinghua University(Science and Technology). 2018, 58(4): 395-401,410 https://doi.org/10.16511/j.cnki.qhdxxb.2018.25.018

References

[1] 陈兴蜀, 赵丹丹, 李辉, 等. 基于虚拟化的不可信模块运行监控[J]. 华中科技大学学报(自然科学版), 2016, 44(3):34-38. CHEN X S, ZHAO D D, LI H, et al. Virtualization-based monitoring of untrusted extensions execution[J]. Journal of Huazhong University of Science and Technology (Natural Science Edition), 2016, 44(3):34-38. (in Chinese)
[2] Wikipedia. Virtual machine escape[EB/OL].[2017-04-01]. http://en.wikipedia.org/wiki/Virtual_machine_escape.
[3] GUAN Q, ZHANG Z, FU S. Ensemble of Bayesian predictors and decision trees for proactive failure management in cloud computing system[J]. Journal of Communications, 2012, 7(1):52-61.
[4] TAN Y, VENKATESH V, GU X. Resilient self-compressive monitoring for large-scale hosting infrastructures[J]. IEEE Transaction on Parallel and Distributed Systems, 2013, 24(3):576-586.
[5] KC K, GU X. ELT:Efficient log-based troubleshooting system for cloud computing infrastructures[C]//Proceedings of 201130th IEEE International Symposium on Reliable Distributed Systems. Madrid, Spain:IEEE, 2011:11-20.
[6] NIKOLAI J, WANG Y. Hypervisor-based cloud intrusion detection system[C]//International Conference on Computing, Networking and Communications. Honolulu, HI, USA:IEEE, 2014:989-993.
[7] 汪圣平, 唐青昊. 一种虚拟机逃逸的防护方法及装置:中国, CN201510958935.5[P]. 2015-12-18. WANG S P, TANG Q H. Protection method and device for virtual machine escape:China, CN201510958935.5[P]. 2015-12-18. (in Chinese)
[8] 栾建海, 汤迪斌, 李常坤, 等. 一种检测虚拟机逃逸的方法及装置:中国, CN201610513980.4[P]. 2016-06-03. LUAN J H, TANG D B, LI C K, et al. Method and device for detecting virtual machine escape:China, CN201610513980.4[P]. 2016-06-03. (in Chinese)
[9] FORREST S, HOFMEYR S A, SOMAYAJI A, et al. A sense of self for UNIX processes[C]//Proceedings of the 1996 IEEE Symposium on Security and Privacy. Oakland, CA, USA:IEEE, 1996:120-128.
[10] PAYNE B D, CARBONE M, SHARIF M, et al. Lares:An architecture for secure active monitoring using virtualization[C]//IEEE Symposium on Security and Privacy, 2008. Washington, DC, USA:IEEE, 2008:233-247.
[11] SHARIF M I, LEE W, CUI W, et al. Secure in-VM monitoring using hardware virtualization[C]//Proceedings of the 16th ACM Conference on Computer and Communications Security. Chicago, Illinois, USA:ACM, 2009:477-487.
[12] HAMID R G, ROYA S S. Toward a policy-based distributed intrusion detection system in cloud computing using data mining approaches[C]//2015 International Congress on Technology, Communication and Knowledge. Mashhad, Iran:IEEE, 2015:412-419.
[13] ILHAME E F, MOHAMMED S, SARA C, et al. The analysis performance of an intrusion detection systems based on neural network[C]//Proceedings of the 4th IEEE International Conference. Tangier, Morocco:IEEE, 2017:145-151.
[14] JHA S, TAN K, Maxion R A. Markov chains, callifiers and intrusion detection[C]//Processings of the 14th IEEE Computer Security Foundations Workshop. Cape Breton, Nova Scotia, Canada:IEEE, 2001:206-219.
[15] 国家信息安全漏洞库. QEMU Floppy Disk Controller缓冲区溢出漏洞[EB/OL].[2015-06-03]. http://www.cnnvd.org.cn/vulnerability/show/cv_cnnvdid/CCNNV-201505-207. China National Vulnerability Database of Information Security. Buffer overflow vulnerability of QEMU Floppy Disk Controller[EB/OL].[2015-06-03]. http://www.cnnvd.org.cn/vulnerability/show/cv_cnnvdid/CCNNV-201505-207.(in Chinese)
[16] 国家信息安全漏洞库. QEMU AMD PC-Net Ⅱ Ethernet Controller CRC Handling缓冲区溢出漏洞[EB/OL].[2015-12-01]. http://www.cnnvd.org.cn/vulnerability/show/cv_cnnvdid/CNNVD-201511-435. China National Vulnerability Database of Information Security. Buffer overflow vulnerability of QEMU AMD PC-Net Ⅱ ethernet controller CRC handling[EB/OL].[2015-12-01]. http://www.cnnvd.org.cn/vulnerability/show/cv_cnnvdid/CNNVD-201511-435. (in Chinese)
[17] 国家信息安全漏洞库. QEMU安全漏洞[EB/OL].[2017-02-09]. http://www.cnnvd.org.cn/vulnerability/show/cv_cnnvdid/CNNVD-201702-234. China National Vulnerability Database of Information Security. Security vulnerability of QEMU[EB/OL].[2017-02-09]. http://www.cnnvd.org.cn/vulnerability/show/cv_cnnvdid/CNNVD-201702-234. (in Chinese)