From automation to intelligence: Survey of research on vulnerability discovery techniques
ZOU Quanchen1, ZHANG Tao1, WU Runpu1, MA Jinxin1, LI Meicong1, CHEN Chen2,3, HOU Changyu4
1. China Information Technology security Evaluation Center, Beijing 100085, China; 2. School of Information and Navigation, Air Force Engineering University, Xi'an 710077, China; 3. School of Cyberspace Security, Beijing University of Posts and Telecommunications, Beijing 100876, China; 4. Beijing Central Security Evaluation Technology Co. Ltd., Beijing 100085, China
Abstract：In recent years, the increasing size and complexity of software packages has led to vulnerability discovery techniques gradually becoming more automatic and intelligent. This paper reviews the search characteristics of both traditional vulnerability discovery techniques and learning-based intelligent vulnerability discovery techniques. The traditional techniques include static and dynamic vulnerability discovery techniques which involve model checking, binary comparisons, fuzzing, symbolic execution and vulnerability exploitability analyses. This paper analyzes the problems of each technique and the challenges for realizing full automation of vulnerability discovery. Then, this paper also reviews machine learning and deep learning techniques for vulnerability discovery that include binary function identification, function similarity detection, test input generation, and path constraint solutions. Some challenges are the security and robustness of machine learning algorithms, algorithm selection, dataset collection, and feature selection. Finally, future research should focus on improving the accuracy and efficiency of vulnerability discovery algorithms and improving the automation and intelligence.
邹权臣, 张涛, 吴润浦, 马金鑫, 李美聪, 陈晨, 侯长玉. 从自动化到智能化:软件漏洞挖掘技术进展[J]. 清华大学学报（自然科学版）, 2018, 58(12): 1079-1094.
ZOU Quanchen, ZHANG Tao, WU Runpu, MA Jinxin, LI Meicong, CHEN Chen, HOU Changyu. From automation to intelligence: Survey of research on vulnerability discovery techniques. Journal of Tsinghua University(Science and Technology), 2018, 58(12): 1079-1094.
[1] CPPCHECK TEAM. Cppcheck software official website[EB/OL].[2018-08-02]. http://cppcheck.sourceforge.net/. [2] WHEELER D A. Flawfinder software official website[EB/OL].[2018-08-02]. https://www.dwheeler.com/flawfinder/. [3] DAHSE J. RIPS software official website[EB/OL].[2018-08-02]. http://rips-scanner.sourceforge.net/. [4] PUGH B, LOSKUTOV A. FindBugs software official website[EB/OL].[2018-08-02]. http://findbugs.sourceforge.net/index.html. [5] C A TECHNIQUES. VeraCode software official website[EB/OL].[2018-08-02]. https://www.veracode.com/. [6] NETWORK DESIGN & MANAGEMENT, INC. Fortify software official website[EB/OL].[2018-08-02]. http://www.ndm.net/sast/hp-fortify-static-code-analyzer. [7] SYNOPSYS, INC. Coverity software official website[EB/OL].[2018-08-02]. https://scan.coverity.com/. [8] CHECKMARX LTD. Checkmarx software official website[EB/OL].[2018-08-02]. https://www.checkmarx.com/. [9] LLVM-ADMIN TEAM. LLVM software official website[EB/OL].[2018-08-02]. https://llvm.org/. [10] LLVM-ADMIN TEAM. Clang software official website[EB/OL].[2018-08-02]. http://clang.llvm.org/. [11] 吴世忠, 郭涛, 董国伟. 软件漏洞分析技术[M]. 北京:科学出版社, 2014. WU S Z, GUO T, DONG G W. The techniques of software vulnerability analysis[M]. Beijing:Science Press, 2014. (in Chinese) [12] JOVANOVIC N, KRUEGEL C, KIRDA E. Pixy:A static analysis tool for detecting web application vulnerabilities[C]//Proceedings of the 2006 IEEE Symposium on Security and Privacy. Oakland, California, USA:IEEE Computer Society, 2006:258-263. [13] BUSH W R, PINCUS J D, SIELAFF D J. A static analyzer for finding dynamic programming errors[J]. Software:Practice and Experience, 2000, 30(7):775-802. [14] SHASTRY B, YAMAGUCHI F, RIECK K, et al. Towards vulnerability discovery using staged program analysis[C]//Proceedings of the 13th International Conference on Detection of Intrusions and Malware, and Vulnerability Assessment. New York, USA:Springer, 2016:78-97. [15] GENS D, SCHMITT S, DAVI L, et al. K-Miner:Uncovering memory corruption in Linux[C]//Proceedings of the 2018 Annual Network and Distributed System Security Symposium (NDSS). San Diego, California, USA:Internet Society, 2018. 10.14722/ndss.2018.23326 [16] CHEN H, WAGNER D. MOPS:An infrastructure for examining security properties of software[C]//Proceedings of the 9th ACM Conference on Computer and Communications Security. Washington, DC, USA:ACM, 2002:235-244. [17] HENZINGER T A, JHALA R, MAJUMDAR R, et al. Software verification with BLAST[C]//International SPIN Workshop on Model Checking of Software. Heidelberg, Berlin:Springer, 2003:235-239. [18] BURCH J, CLARKE E M, Long D. Symbolic model checking with partitioned transition relations[M]. Carnegie-Mellon University. Department of Computer Science, 1991. [19] BALAKRISHNAN G, REPS T. WYSINWYX:What you see is not what you execute[J]. ACM Transactions on Programming Languages and Systems (TOPLAS), 2010, 32(6):23. [20] CIFUENTES C, VAN EMMERIK M. Recovery of jump table case statements from binary code[C]//International Workshop on Program Comprehension. Pittsburgh, Pennsylvania, USA:IEEE Computer Society, 1999:192-199. [21] KINDER J, VEITH H. Jakstab:A static analysis platform for binaries[C]//International Conference on Computer Aided Verification. Princeton, USA:Springer, 2008:423-427. [22] KRUEGEL C, ROBERTSON W, VALEUR F, et al. Static disassembly of obfuscated binaries[C]//USENIX Security Symposium. San Diego, CA USA:USENIX Association, 2004(13):18-18. [23] SCHWARZ B, DEBRAY S, Andrews G. Disassembly of executable code revisited[C]//Proceedings of the Ninth Working Conference on Reverse Engineering. Richmond, VA, USA:IEEE Computer Society, 2002:45-54. [24] TROGER J, CIFUENTES C. Analysis of virtual method invocation for binary translation[C]//Proceedings of the Ninth Working Conference on Reverse Engineering. Richmond, VA, USA:IEEE Computer Society, 2002:65-74. [25] XU L, SUN F, SU Z. Constructing precise control flow graphs from binaries[R]. University of California, Davis:2009. [26] FEIST J, MOUNIER L, POTET M L. Statically detecting use after free on binary code[J]. Journal of Computer Virology and Hacking Techniques, 2014, 10(3):211-217. [27] CHENG S, YANG J, WANG J, et al. Loongchecker:Practical summary-based semi-simulation to detect vulnerability in binary code[C]//Proceedings of the 2011 IEEE 10th International Conference on Trust, Security and Privacy in Computing and Communications. Washington, DC, USA:IEEE Computer Society, 2011:150-159. [28] GOTOVCHITS I, VAN Tonder R, BRUMLEY D. Saluki:Finding taint-style vulnerabilities with static property checking[C]//Network and Distributed Systems Security (NDSS) Symposium. San Diego, CA, USA:Internet Society, 2018. 10.14722/bar.2018.23019. [29] SHA L, FU J, JING C, et al. PVDF:An automatic patch-based vulnerability description and fuzzing method[C]//Communications Security Conference. Beijing, China:IET, 2014:1-8. [30] GAO D, REITER M K, SONG D. BinHunt:Automatically finding semantic differences in binary programs[C]//International Conference on Information and Communications Security. Birmingham UK:Springer, 2008:238-255. [31] GOOGLE INC. AFL software official website[EB/OL].[2018-08-02]. http://lcamtuf.coredump.cx/afl/ [32] RAWAT S, JAIN V, KUMAR A, et al. Vuzzer:Application-aware evolutionary fuzzing[C]//Proceedings of the Network and Distributed System Security Symposium (NDSS). San Diego, CA, USA:Internet Society, 2017. 10.14722/ndss.2017.23404. [33] GOOGLE INC. Honggfuzz software official website[EB/OL].[2018-08-02]. http://honggfuzz.com [34] GOOGLE INC. LibFuzzer software official website[EB/OL].[2018-08-02]. https://github.com/Dor1s/libfuzzer-workshop. [35] LI Y, CHEN B, CHANDRAMOHAN M, et al. Steelix:Program-state based binary fuzzing[C]//Joint Meeting on Foundations of Software Engineering. Paderborn, Germany:ACM, 2017:627-637. [36] PENG H, SHOSHITAISHVILI Y, PAYER M, T-Fuzz:Fuzzing by program transformation[C]//IEEE Symposium on Security and Privacy (SP). San Francisco, CA, USA:IEEE Computer Society, 2018:697-710. [37] BÖHME M, PHAM V T, ROYCHOUDHURY A. Coverage-based greybox fuzzing as Markov chain[C]//Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security. Vienna, Austria:ACM, 2016:1032-1043. [38] BÖHME M, PHAM V T, NGUYEN M D, et al. Directed greybox fuzzing[C]//Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security. Dallas, TX, USA:ACM, 2017:2329-2344. [39] STEPHENS N, GROSEN J, SALLS C, et al. Driller:Augmenting fuzzing through selective symbolic execution[C]//Proceedings of the Network and Distributed System Security Symposium. San Diego, California, USA:Internet Society, 2016:1-16. [40] GAN S, ZHANG C, QIN X, et al, CollAFL:Path sensitive fuzzing[C]//2018 IEEE Symposium on Security and Privacy (SP). San Fransisco, CA, USA:IEEE Computer Society, 2018:660-677. [41] GANESH V, LEEK T, RINARD M. Taint-based directed whitebox fuzzing[C]//International Conference on Software Engineering. British Columbia, Canada:IEEE, 2009:474-484. [42] WANG T, WEI T, GU G, et al. TaintScope:A checksum-aware directed fuzzing tool for automatic software vulnerability detection[C]//IEEE Symposium on Security and Privacy. Oakland, California, USA:IEEE Computer Society, 2010:497-512. [43] DOLAN-GAVITT B, HULIN P, KIRDA E, et al. Lava:Large-scale automated vulnerability addition[C]//IEEE Symposium on Security and Privacy. San Jose, California, USA:IEEE Computer Society, 2016:110-121. [44] PEACH TECH. Peach software official website[EB/OL].[2018-08-02]. http://www.peachfuzzer.com/products/peach-platform/ [45] BRADSHAW S. Spike software official website[EB/OL].[2018-08-02]. http://www.immunitysec.com/ [46] PHAM V T, BÖHME M, ROYCHOUDHURY A. Model-based whitebox fuzzing for program binaries[C]//Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering. Singapore:ACM, 2016:543-553. [47] YANG X, CHEN Y, EIDE E, et al. Finding and understanding bugs in C compilers[J]. ACM SIGPLAN Notices, 2011, 46(6):283-294. [48] HOLLER C, HERZIG K, ZELLER A. Fuzzing with code fragments[C]//USENIX Security Symposium. San Francisco, California, USA:USENIX Association, 2012:445-458. [49] VEGGALAM S, RAWAT S, HALLER I, et al. Ifuzzer:An evolutionary interpreter fuzzer using genetic programming[C]//European Symposium on Research in Computer Security. Heraklion, Greece:Springer, 2016:581-601. [50] RUDERSMAN J. Jsfunfuzz software official website[EB/OL].[2018-08-02].http://www.squarefree.com/2007/08/02/introducing-jsfunfuzz/ [51] DEWEY K, ROESCH J, HARDEKOPF B. Language fuzzing using constraint logic programming[C]//Proceedings of the 29th ACM/IEEE international conference on Automated software engineering. Vasteras, Sweden:ACM, 2014:725-730. [52] WOO M, SANG K C, GOTTLIEB S, et al. Scheduling black-box mutational fuzzing[C]//Proceedings of the 2013 ACM SIGSAC Conference on Computer & Communications Security. Berlin, Germany:ACM, 2013:511-522. [53] REBERT A, CHA S K, AVGERINOS T, et al. Optimizing seed selection for fuzzing[C]//USENIX Security Symposium. San Jose, California, USA:USENIX Association, 2014:861-875. [54] WANG S, NAM J, TAN L. QTEP:Quality-aware test case prioritization[C]//Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering. Paderborn, Germany:ACM, 2017:523-534. [55] PETSIOS T, ZHAO J, KEROMYTIS A D, et al. Slowfuzz:Automated domain-independent detection of algorithmic complexity vulnerabilities[C]//Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security. Dallas, TX, USA:ACM, 2017:2155-2168. [56] BOYER R S, ELSPAS B, LEVITT K N. SELECT-A formal system for testing and debugging programs by symbolic execution[J]. ACM SigPlan Notices, 1975, 10(6):234-245. [57] CLARKE L A. A program testing system[C]//Proceedings of the 1976 annual conference. Texas, USA:ACM, 1976:488-491. [58] HOWDEN W E. Symbolic testing and the DISSECT symbolic evaluation system[J]. IEEE Transactions on Software Engineering, 1977(4):266-278. [59] KING J C. Symbolic execution and program testing[J]. Communications of the ACM, 1976, 19(7):385-394. [60] GODEFROID P, LEVIN M Y, Molnar D A. Automated whitebox fuzz testing[C]//Proceedings of the Network and Distributed System Security Symposium (NDSS). San Diego, California, USA:Internet Society, 2008, 8:151-166. [61] CHIPOUNOV V, KUZNETSOV V, CANDEA G. S2E:A platform for in-vivo multi-path analysis of software systems[J]. ACM Sigarch Computer Architecture News, 2011, 47(4):265-278. [62] CHA S K, AVGERINOS T, Rebert A, et al. Unleashing mayhem on binary code[C]//IEEE Symposium on Security and Privacy (SP). San Francisco, California, USA:IEEE,Computer Society, 2012:380-394. [63] CADAR C, DUNBAR D, ENGLER D R. KLEE:Unassisted and automatic generation of high-coverage tests for complex systems programs[C]//USENIX Conference on Operating Systems Design and Implementation. San Diego USA:USENIX Association, 2009:209-224. [64] SAUDEL F, SALWAN J. Triton:A dynamic symbolic execution framework[C]//Symposium sur la sécurité des technologies de l'information et des communications. Rennes, France:SSTIC, 2015:31-54. [65] SHOSHITAISHVILI Y, KRUEGEL C, VIGNA G, et al. Sok:(state of) the art of war:Offensive techniques in binary analysis[C]//2016 IEEE Symposium on Security and Privacy (SP). San Jose, California, USA:IEEE Computer Society, 2016:138-157. [66] SHOSHITAISHVILI Y, WANG R, HAUSER C, et al. Firmalice-automatic detection of authentication bypass vulnerabilities in binary firmware[C]//Proceedings of the Network and Distributed System Security Symposium (NDSS). San Diego, California, USA:Internet Society, 2015. 10.14722/ndss.2015.23294. [67] AVGERINOS T, REBERT A, Cha S K, et al. Enhancing symbolic execution with veritesting[C]//Proceedings of the 36th International Conference on Software Engineering. Hyderabad, India:ACM, 2014:1083-1094. [68] MA K K, PHANG K Y, FOSTER J S, et al. Directed symbolic execution[C]//International Static Analysis Symposium. Heidelberg, Berlin:Springer, 2011:95-111. [69] GODEFROID P, NORI A V, Rajamani S K, et al. Compositional may-must program analysis:Unleashing the power of alternation[J]//ACM Sigplan Notices, 2010, 45(1):43-56. [70] RAMOS D A, ENGLER D R. Under-constrained symbolic execution:Correctness checking for real code[C]//USENIX Security Symposium. Washington, D.C., USA:USENIX Association, 2015:49-64. [71] BOONSTOPPEL P, CADAR C, ENGLER D. RWset:Attacking path explosion in constraint-based test generation[C]//International Conference on Tools and Algorithms for the Construction and Analysis of Systems. Budapest, Hungary:Springer, 2008:351-366. [72] BORRALLERAS C, LUCAS S, OLIVERAS A, et al. SAT modulo linear arithmetic for solving polynomial constraints[J]. Journal of Automated Reasoning, 2012, 48(1):107-131. [73] ARMANDO A, BONACINA M P, RANISE S, et al. New results on rewrite-based satisfiability procedures[J]. ACM Transactions on Computational Logic (TOCL), 2009, 10(1):4. [74] CIMATTI A, GRIGGIO A, SCHAAFSMA B J, et al. The mathSAT5 SMT solver[C]//International Conference on Tools and Algorithms for the Construction and Analysis of Systems. Rome, Italy:Springer, 2013:93-107. [75] JHA S, LIMAYE R, SESHIA S A. Beaver:Engineering an efficient smt solver for bit-vector arithmetic[C]//International Conference on Computer Aided Verification. Heidelberg, Berlin:Springer, 2009:668-674. [76] VAN K T, OGAWA M. SMT for polynomial constraints on real numbers[J]. Electronic Notes in Theoretical Computer Science, 2012, 289:27-40. [77] SEN K, MARINOV D, AGHA G. CUTE:A concolic unit testing engine for C[J]//ACM SIGSOFT Software Engineering Notes. ACM, 2005, 30(5):263-272. [78] VISSER W, GELDENHUYS J, DWYER M B. Green:Reducing, reusing and recycling constraints in program analysis[C]//Proceedings of the ACM SIGSOFT 20th International Symposium on the Foundations of Software Engineering. Cary, NC, USA:ACM, 2012:1-11. [79] AQUINO A, BIANCHI F A, CHEN M, et al. Reusing constraint proofs in program analysis[C]//Proceedings of the 2015 International Symposium on Software Testing and Analysis. Baltimore, MD, USA:ACM, 2015:305-315. [80] JIA X, GHEZZI C, YING S. Enhancing reuse of constraint solutions to improve symbolic execution[C]//Proceedings of the 2015 International Symposium on Software Testing and Analysis. Baltimore, MD, USA:ACM, 2015:177-187. [81] YANG G, PǍSǍREANU C S, KHURSHID S. Memoized symbolic execution[C]//Proceedings of the 2012 International Symposium on Software Testing and Analysis. Minneapolis, MN, USA:ACM, 2012:144-154. [82] CADAR C, GANESH V, PAWLOWSKI P M, et al. EXE:Automatically generating inputs of death[J]. ACM Transactions on Information and System Security (TISSEC), 2008, 12(2):10. [83] AVGERINOS T, CHA S K, REBERT A, et al. Automatic exploit generation[J]. Communications of the ACM, 2014, 57(2):74-84. [84] BUCUR S, URECHE V, ZAMFIR C, et al. Parallel symbolic execution for automated real-world software testing[C]//Proceedings of the sixth conference on Computer systems. Salzburg, Austria:ACM, 2011:183-198. [85] MICORSOFT INC. !exploitable software official website[EB/OL].[2018-08-02]. https://archive.codeplex.com/?p=msecdbg [86] Software Engineering Institute, Carnegie-Mellon University. gdb-exploitable software official website[EB/OL].[2018-08-02]. https://github.com/jfoote/exploitable [87] Google Inc. ASan software official website[EB/OL].[2018-08-02]. https://github.com/google/sanitizers [88] BRUMLEY D, POOSANKAM P, SONG D, et al. Automatic patch-based exploit generation is possible:Techniques and implications[C]//IEEE Symposium on Security and Privacy. Oakland, California, USA:IEEE Computer Society, 2008:143-157. [89] HEELAN S. Automatic generation of control flow hijacking exploits for software vulnerabilities[D]. Oxford, UK:University of Oxford, 2009. [90] CHA S K, AVGERINOS T, REBERT A, et al. Unleashing mayhem on binary code[C]//IEEE Symposium on Security and Privacy (SP). San Francisco, California, USA:IEEE Computer Society, 2012:380-394. [91] HU H, CHUA Z L, ADRIAN S, et al. Automatic generation of data-oriented exploits[C]//USENIX Security Symposium. Washington, D.C., USA:USENIX Association, 2015:177-192. [92] LECUN Y, BENGIO Y, HINTON G. Deep learning[J]. Nature, 2015, 521(7553):436-444. [93] BAO T, BURKET J, WOO M, et al. BYTEWEIGHT:Learning to recognize functions in binary code[C]//USENIX Security Symposium. San Jose, California, USA:USENIX Association, 2014:845-860. [94] CYLAB SECURITY AND PRIVACY INSTITUTE. BAP software official website[EB/OL].[2018-08-02]. http://bap.ece.cmu.edu/. [95] SHIN E C R, SONG D, MOAZZEZI R. Recognizing functions in binaries with neural networks[C]//USENIX Security Symposium. Washington, D.C., USA:USENIX Association, 2015:611-626. [96] CHUA Z L, SHEN S, SAXENA P, et al. Neural nets can learn function type signatures from binaries[C]//USENIX Security Symposium. Vancouver, BC, Canada:USENIX Association, 2017:99-115. [97] XU X, LIU C, FENG Q, et al. Neural network-based graph embedding for cross-platform binary code similarity detection[C]//Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security. Dallas, TX, USA:ACM, 2017:363-376. [98] FENG Q, ZHOU R, XU C, et al. Scalable graph-based bug search for firmware images[C]//Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security. Vienna, Austria:ACM, 2016:480-491. [99] GODEFROID P, PELEG H, SINGH R. Learn&fuzz:Machine learning for input fuzzing[C]//Proceedings of the 32nd IEEE/ACM International Conference on Automated Software Engineering. Urbana, IL, USA:IEEE Press, 2017:50-59. [100] SHE D, PEI K, EPSTEIN D, et al. NEUZZ:Efficient fuzzing with neural program learning[J/OL]. (2018-07-15). https://arxiv.org/abs/1807.05620. [101] BÖTTINGER K, GODEFROID P, SINGH R. Deep reinforcement fuzzing[J/OL]. (2018-01-14). https://arxiv.org/abs/1801.04589. [102] NICHOLS N, RAUGAS M, JASPER R, et al. Faster fuzzing:Reinitialization with deep neural models[J/OL]. (2017-11-08). https://arxiv.org/abs/1711.02807. [103] RAJPAL M, BLUM W, SINGH R. Not all bytes are equal:Neural byte sieve for fuzzing[J/OL]. (2017-11-10). https://arxiv.org/abs/1711.04596. [104] SPIEKER H, GOTLIEB A, MARIJAN D, et al. Reinforcement learning for automatic test case prioritization and selection in continuous integration[C]//Proceedings of the 26th ACM SIGSOFT International Symposium on Software Testing and Analysis. Santa Barbara, CA, USA:ACM, 2017:12-22. [105] CHEN P, CHEN H. Angora:Efficient fuzzing by principled search[C]//IEEE Symposium on Security and Privacy. San Francisco, CA, USA:IEEE Computer Society, 2018:758-772 [106] GRIECO G, GRINBLAT G L, UZAL L, et al. Toward large-scale vulnerability discovery using machine learning[C]//Proceedings of the Sixth ACM Conference on Data and Application Security and Privacy. New Orleans, LA, USA:ACM, 2016:85-96. [107] LI Z, ZOU D, XU S, et al. VulDeePecker:A deep learning-based system for vulnerability Detection[C]//Network and Distributed Systems Security (NDSS) Symposium. San Diego, California USA:Internet Society, 2018. 10.14722/ndss.2018.23158. [108] HOUSEHOLDER A D, FOOTE J M. Probability-based parameter selection for black-box fuzz testing[R]. Pittsburgh, PA:Software Engineering Institute, Carnegie Mellon University, 2012. [109] YAN G, KUCUK Y, SLOCUM M, et al. A Bayesian cognitive approach to quantifying software exploitability based on reachability testing[C]//International Conference on Information Security. Honolulu, USA:Springer, 2016:343-365. [110] YAN G, LU J, SHU Z, et al. ExploitMeter:Combining fuzzing with machine learning for automated evaluation of software exploitability[C]//2017 IEEE Symposium on Privacy-Aware Computing (PAC). Washington DC, USA:IEEE, 2017:164-175. [111] MARCUS G. Deep Learning:A critical appraisal[J/OL]. (2018-10-02). https://arxiv.org/abs/1801.00631. [112] STOICA I, SONG D, POPA R A, et al. A berkeley view of systems challenges for AI[J/OL]. (2017-12-15). https://arxiv.org/abs/1712.05855. [113] STEVENS R, SUCIU O, RUEF A, et al. Summoning demons:The pursuit of exploitable bugs in machine learning[J/OL]. (2017-01-17). https://arxiv.org/abs/1701.04739. [114] GOODFELLOW I J, SHLENS J, SZEGEDY C. Explaining and harnessing adversarial examples[J]. Computer Science, 2014 [115] GEHR T, MIRMAN M, DRACHSLER-COHEN D, et al. AI 2:Safety and robustness certification of neural networks with abstract interpretation[C]//IEEE Symposium on Security and Privacy (SP). San Francisco, CA, USA:IEEE Computer Society, 2018. 10.1109/SP.2018.00058 [116] CARLINI N, WAGNER D. Towards evaluating the robustness of neural networks[C]///IEEE Symposium on Security and Privacy (SP). San Jose, CA, USA:IEEE Computer Society, 2017:39-57 [117] HUANG X, KWIATKOWSKA M, WANG S, et al. Safety verification of deep neural networks[C]//International Conference on Computer Aided Verification. Heidelberg:Springer, 2017:3-29. [118] PEI K, CAO Y, YANG J, et al. Deepxplore:Automated whitebox testing of deep learning systems[C]//Proceedings of the 26th Symposium on Operating Systems Principles. Shanghai, China:ACM, 2017:1-18. [119] JAGIELSKI M, OPREA A, BIGGIO B, et al. Manipulating machine learning:Poisoning attacks and countermeasures for regression learning[C]//IEEE Symposium on Security and Privacy (SP). San Francisco, CA, USA:IEEE Computer Society, 2018. [120] LI B, WANG Y, SINGH A, et al. Data poisoning attacks on factorization-based collaborative filtering[C]//Advances in Neural Information Processing Systems. Barcelona, Spain:NIPS Press, 2016:1885-1893. [121] YANG G, GONG N Z, CAI Y. Fake co-visitation injection attacks to recommender systems[C]//Network and Distributed System Security Symposium. San Diego, California, USA:Internet Society, 2017. [122] KE L, LI B, VOROBEYCHIK Y. Behavioral experiments in email filter evasion[C]//Thirtieth AAAI Conference on Artificial Intelligence. Arizona, USA:AAAI Press, 2016:827-833. [123] LIU Y, CHEN X, LIU C, et al. Delving into transferable adversarial examples and black-box attacks[J/OL]. (2016-11-08). https://arxiv.org/abs/1611.02770. [124] PAPERNOT N, Mcdaniel P, GOODFELLOW I, et al. Practical black-box attacks against machine learning[C]//Proceedings of the 2017 ACM on Asia Conference on Computer and Communications Security. Abu Dhabi, United Arab Emirates:ACM, 2017:506-519. [125] XU W, QI Y, EVANS D. Automatically evading classifiers[C]//Proceedings of the 2016 Network and Distributed Systems Symposium. San Diego, California, USA:Internet Society, 2016. 10.14722/ndss.2016.23115. [126] FREDRIKSON M, JHA S, RISTENPART T. Model inversion attacks that exploit confidence information and basic countermeasures[C]//Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security. Denver, USA:ACM, 2015:1322-1333. [127] FREDRIKSON M, LANTZ E, JHA S, et al. Privacy in pharmacogenetics:An end-to-end case study of personalized warfarin dosing[C]//USENIX Security Symposium. San Jose, California, USA:USENIX Association, 2014:17-32. [128] LOWD D, MEEK C. Adversarial learning[C]//Proceedings of the Eleventh ACM SIGKDD International Conference on Knowledge Discovery in Data Mining. Chicago, IL, USA:ACM, 2005:641-647. [129] TRAMÈR F, ZHANG F, JUELS A, et al. Stealing machine learning models via prediction APIs[C]//USENIX Security Symposium. Austin, TX, USA:USENIX Association, 2016:601-618. [130] WANG B, GONG N Z. Stealing hyperparameters in machine learning[C]//IEEE Symposium on Security and Privacy. San Francisco, CA, USA:IEEE Computer Society, 2018. [131] DAUPHIN Y N, FAN A, AULI M, et al. Language modeling with gated convolutional networks[J/OL]. (2016-12-23). https://arxiv.org/abs/1612.08083. [132] YIN W, KANN K, YU M, et al. Comparative study of CNN and RNN for natural language processing[J/OL]. (2017-02-07). https://arxiv.org/abs/1702.01923. [133] WANG J, CHEN B, Wei L, et al. Skyfire:Data-driven seed generation for fuzzing[C]//IEEE Symposium on Security and Privacy (SP). San Jose, CA, USA:IEEE Computer Society, 2017:579-594.
2018, Vol. 58 
