| ELECTRONIC ENGINEERING |
|
|
|
|
|
Anomaly detection in surveillance videos: A survey |
| WANG Zhiguo, ZHANG Yujin |
| Image Engineering Laboratory, Department of Electronic Engineering, Tsinghua University, Beijing 100084, China |
|
|
|
|
Abstract Surveillance videos are important for maintaining social welfare. This paper classifies and summarizes the traditional and advanced video anomaly detection algorithms. First, the algorithms are classified into different classes according to their development stages, model categories and detection criteria and then they are summarized by class. Then, the advantages and the disadvantages of the different algorithms are identified by comparing the algorithms belonging to different classes. This paper specifically analyses the characteristics of the cluster criterion and the reconstruction criterion in different development stages. Finally, this paper identifies the commonly used model assumptions and the domain knowledge and summarizes the accuracies of the various algorithms. Future research directions are also discussed.
|
| Keywords
surveillance video
anomaly detection
deep learning
machine learning
algorithm comparison
|
|
Issue Date: 27 April 2020
|
|
|
[1] CHANDOLA V, BANERJEE A, KUMAR V. Anomaly detection:A survey[J]. ACM Computing Surveys, 2009, 41(3):15.
[2] CHONG Y S, TAY Y H. Modeling representation of videos for anomaly detection using deep learning:A review[Z/OL]. arXiv:1505.00523, 2015.
[3] IONESCU R T, SMEUREANU S, POPESCU M, et al. Detecting abnormal events in video using narrowed normality clusters[C]//Proceedings of 2019 IEEE Winter Conference on Applications of Computer Vision. Hawaii, USA, 2019:1951-1960.
[4] LIU W, LUO W X, LIAN D Z, et al. Future frame prediction for anomaly detection:A new baseline[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA, 2018:6536-6545.
[5] POPOOLA O P, WANG K J. Video-based abnormal human behavior recognition:A review[J]. IEEE Transactions on Systems, Man, and Cybernetics, Part C:Applications and Reviews, 2012, 42(6):865-878.
[6] LI T, CHANG H, WANG M, et al. Crowded scene analysis:A survey[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2015, 25(3):367-386.
[7] AHMED S A, DOGRA D P, KAR S, et al. Trajectory-based surveillance analysis:A survey[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2019, 29(7):1985-1997.
[8] KIRAN B R, THOMAS D M, PARAKKAL R. An overview of deep learning based methods for unsupervised and semi-supervised anomaly detection in videos[J]. Journal of Imaging, 2018, 4(2):36.
[9] CHALAPATHY R, CHAWLA S. Deep learning for anomaly detection:A survey[Z/OL]. arXiv:1901.03407, 2019.
[10] CHENG K W, CHEN Y T, FANG W H. Video anomaly detection and localization using hierarchical feature representation and Gaussian process regression[C]//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston, USA, 2015:2909-2917.
[11] DEL GIORNO A, ANDREW BAGNELL J, HEBERT M. A discriminative framework for anomaly detection in large videos[C]//Proceedings of the 14th European Conference on Computer Vision. Amsterdam, Netherlands, 2016:334-349.
[12] IONESCU R T, SMEUREANU S, ALEXE B, et al. Unmasking the abnormal events in video[C]//Proceedings of 2017 IEEE International Conference on Computer Vision. Venice, Italy, 2017:2914-2922.
[13] XU Y, OUYANG X, CHENG Y, et al. Dual-mode vehicle motion pattern learning for high performance road traffic anomaly detection[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Salt Lake City, USA, 2018:145-152.
[14] CAI Y F, WANG H, CHEN X B, et al. Trajectory-based anomalous behaviour detection for intelligent traffic surveillance[J]. IET Intelligent Transport Systems, 2015, 9(8):810-816.
[15] ZHOU X G, ZHANG L Q. Abnormal event detection using recurrent neural network[C]//Proceedings of 2015 International Conference on Computer Science and Applications. Wuhan, China, 2015:222-226.
[16] KIM J, GRAUMAN K. Observe locally, infer globally:A space-time MRF for detecting abnormal activities with incremental updates[C]//Proceedings of 2009 IEEE Conference on Computer Vision and Pattern Recognition. Miami, USA, 2009:2921-2928.
[17] THIDA M, ENG H L, REMAGNINO P. Laplacian eigenmap with temporal constraints for local abnormality detection in crowded scenes[J]. IEEE Transactions on Cybernetics, 2013, 43(6):2147-2156.
[18] SINGH D, KRISHNA MOHAN C. Graph formulation of video activities for abnormal activity recognition[J]. Pattern Recognition, 2017, 65:265-272.
[19] XU J X, DENMAN S, SRIDHARAN S, et al. Dynamic texture reconstruction from sparse codes for unusual event detection in crowded scenes[C]//Proceedings of 2011 Joint ACM Workshop on Modeling and Representing Events. Scottsdale, USA, 2011:25-30.
[20] LEE D G, SUK H I, PARK S K, et al. Motion influence map for unusual human activity detection and localization in crowded scenes[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2015, 25(10):1612-1623.
[21] LEYVA R, SANCHEZ V, LI C T. Fast detection of abnormal events in videos with binary features[C]//Proceedings of 2018 IEEE International Conference on Acoustics, Speech and Signal Processing. Calgary, Canada, 2018:1318-1322.
[22] 黄鑫, 肖世德, 宋波. 监控视频中的车辆异常行为检测[J]. 计算机系统应用, 2018, 27(2):125-131. HUANG X, XIAO S D, SONG B. Detection of vehicle's abnormal behaviors in surveillance video[J]. Computer Systems & Applications, 2018, 27(2):125-131. (in Chinese)
[23] YIN J, YANG Q, PAN J J. Sensor-based abnormal human-activity detection[J]. IEEE Transactions on Knowledge and Data Engineering, 2008, 20(8):1082-1090.
[24] SALIGRAMA V, CHEN Z. Video anomaly detection based on local statistical aggregates[C]//Proceedings of 2012 IEEE Conference on Computer Vision and Pattern Recognition. Providence, USA, 2012:2112-2119.
[25] HINAMI R, MEI T, SATOH S. Joint detection and recounting of abnormal events by learning deep generic knowledge[C]//Proceedings of 2017 IEEE International Conference on Computer Vision. Venice, Italy, 2017:3639-3647.
[26] MEHRAN R, OYAMA A, SHAH M. Abnormal crowd behavior detection using social force model[C]//Proceedings of 2009 IEEE Conference on Computer Vision and Pattern Recognition. Miami, USA, 2009:935-942.
[27] HASSNER T, ITCHER Y, KLIPER-GROSS O. Violent flows:Real-time detection of violent crowd behavior[C]//Proceedings of 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops. Providence, USA, 2012:1-6.
[28] MOHAMMADI S, PERINA A, KIANI H, et al. Angry crowds:Detecting violent events in videos[C]//Proceedings of the 14th European Conference on Computer Vision. Amsterdam, Netherlands, 2016:3-18.
[29] HU X, HUANG Y P, GAO X M, et al. Squirrel-cage local binary pattern and its application in video anomaly detection[J]. IEEE Transactions on Information Forensics and Security, 2019, 14(4):1007-1022.
[30] HU D H, ZHANG X X, YIN J, et al. Abnormal activity recognition based on HDP-HMM models[C]//Proceedings of the 21st International Jont Conference on Artifical Intelligence. Pasadena, USA:Morgan Kaufmann Publishers, 2009:1715-1720.
[31] YANG M Y, LIAO W T, CAO Y P, et al. Video event recognition and anomaly detection by combining Gaussian process and hierarchical Dirichlet process models[J]. Photogrammetric Engineering & Remote Sensing, 2018, 84(4):203-214.
[32] CANDÉS E, LI X D, MA Y, et al. Robust principal component analysis?:Recovering low-rank matrices from sparse errors[C]//Proceedings of 2010 IEEE Sensor Array and Multichannel Signal Processing Workshop. Jerusalem, Israel, 2010:201-204.
[33] XIONG L, CHEN X, SCHNEIDER J. Direct robust matrix factorizatoin for anomaly detection[C]//Proceedings of the 2011 IEEE 11th International Conference on Data Mining. Vancouver, Canada, 2011:844-853.
[34] DEBRUYNE M, VERDONCK T. Robust kernel principal component analysis and classification[J]. Advances in Data Analysis and Classification, 2010, 4(2-3):151-167.
[35] ZHU X B, LIU J, WANG J Q, et al. Sparse representation for robust abnormality detection in crowded scenes[J]. Pattern Recognition, 2014, 47(5):1791-1799.
[36] LUO W X, LIU W, GAO S H. A revisit of sparse coding based anomaly detection in stacked RNN framework[C]//Proceedings of 2017 IEEE International Conference on Computer Vision. Venice, Italy, 2017:341-349.
[37] ZHAO B, LI F F, XING E P. Online detection of unusual events in videos via dynamic sparse coding[C]//Proceedings of CVPR 2011. Providence, USA, 2011:3313-3320.
[38] YUAN Y, FENG Y C, LU X Q. Structured dictionary learning for abnormal event detection in crowded scenes[J]. Pattern Recognition, 2018, 73:99-110.
[39] CONG Y, YUAN J S, LIU J. Abnormal event detection in crowded scenes using sparse representation[J]. Pattern Recognition, 2013, 46(7):1851-1864.
[40] CONG Y, YUAN J S, LIU J. Sparse reconstruction cost for abnormal event detection[C]//Proceedings of CVPR 2011. Providence, USA, 2011:3449-3456.
[41] LU C W, SHI J P, JIA J Y. Abnormal event detection at 150 FPS in MATLAB[C]//Proceedings of 2013 IEEE International Conference on Computer Vision. Sydney, Australia, 2013:2720-2727.
[42] REN H M, LIU W F, OLSEN S I, et al. Unsupervised behavior-specific dictionary learning for abnormal event detection[C]//Proceedings of 2015 British Machine Vision Conference. Swansea, UK:BMVA Press, 2015:28.1-28.13.
[43] MAHADEVAN V, LI W X, BHALODIA V, et al. Anomaly detection in crowded scenes[C]//Proceedings of 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. San Francisco, USA, 2010:1975-1981.
[44] LI W X, MAHADEVAN V, VASCONCELOS N. Anomaly detection and localization in crowded scenes[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014, 36(1):18-32.
[45] PRUTEANU-MALINICI I, CARIN L. Infinite hidden Markov models for unusual-event detection in video[J]. IEEE Transactions on Image Processing, 2008, 17(5):811-822.
[46] XIANG T, GONG S G. Incremental and adaptive abnormal behaviour detection[J]. Computer Vision and Image Understanding, 2008, 111(1):59-73.
[47] KRATZ L, KO N. Anomaly detection in extremely crowded scenes using spatio-temporal motion pattern models[C]//Proceedings of 2009 IEEE Conference on Computer Vision and Pattern Recognition. Miami, USA, 2009:1446-1453.
[48] WANG B, YE M, LI X, et al. Abnormal crowd behavior detection using high-frequency and spatio-temporal features[J]. Machine Vision and Applications, 2012, 23(3):501-511.
[49] HOSPEDALES T, GONG S G, XIANG T. A Markov clustering topic model for mining behaviour in video[C]//Proceedings of the 2009 IEEE 12th International Conference on Computer Vision. Kyoto, Japan, 2009:1165-1172.
[50] SONG L, JIANG F, SHI Z K, et al. Toward dynamic scene understanding by hierarchical motion pattern mining[J]. IEEE Transactions on Intelligent Transportation Systems, 2014, 15(3):1273-1285.
[51] 李娟, 张冰怡, 冯志勇, 等. 基于隐马尔可夫模型的视频异常场景检测[J]. 计算机工程与科学, 2017, 39(7):1300-1308. LI J, ZHANG B Y, FENG Z Y, et al. Anomaly detection based on hidden Markov model in videos[J]. Computer Engineering and Science, 2017, 39(7):1300-1308. (in Chinese)
[52] QIAO M N, WANG T, LI J K, et al. Abnormal event detection based on deep autoencoder fusing optical flow[C]//Proceedings of the 2017 36th Chinese Control Conference. Dalian, China, 2017:11098-11103.
[53] TRAN H, HOGG D C. Anomaly detection using a convolutional winner-take-all autoencoder[C]//Proceedings of British Machine Vision Conference 2017. London, UK, 2017.
[54] XU D, RICCI E, YAN Y, et al. Learning deep representations of appearance and motion for anomalous event detection[Z/OL]. arXiv:1510.01553, 2015.
[55] XU D, YAN Y, RICCI E, et al. Detecting anomalous events in videos by learning deep representations of appearance and motion[J]. Computer Vision and Image Understanding, 2017, 156:117-127.
[56] SMEUREANU S, IONESCU R T, POPESCU M, et al. Deep appearance features for abnormal behavior detection in video[C]//Proceedings of the 19th International Conference on Image Analysis and Processing. Catania, Italy, 2017:779-789.
[57] NAZARE T S, DE MELLO R F, PONTI M A. Are pre-trained CNNs good feature extractors for anomaly detection in surveillance videos?[Z/OL]. arXiv:1811.08495, 2018.
[58] SABOKROU M, FATHY M, HOSEINI M, et al. Real-time anomaly detection and localization in crowded scenes[C]//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops. Boston, USA, 2015:56-62.
[59] SABOKROU M, FAYYAZ M, FATHY M, et al. Deep-cascade:Cascading 3D deep neural networks for fast anomaly detection and localization in crowded scenes[J]. IEEE Transactions on Image Processing, 2017, 26(4):1992-2004.
[60] SABOKROU M, FATHY M, MOAYED Z, et al. Fast and accurate detection and localization of abnormal behavior in crowded scenes[J]. Machine Vision and Applications, 2017, 28(8):965-985.
[61] SABOKROU M, FAYYAZ M, FATHY M, et al. Deep-anomaly:Fully convolutional neural network for fast anomaly detection in crowded scenes[J]. Computer Vision and Image Understanding, 2018, 172:88-97.
[62] FENG Y C, YUAN Y, LU X Q. Learning deep event models for crowd anomaly detection[J]. Neurocomputing, 2017, 219:548-556.
[63] SUN J Y, WANG X Z, XIONG N X, et al. Learning sparse representation with variational auto-encoder for anomaly detection[J]. IEEE Access, 2018, 6:33353-33361.
[64] CHU W Q, XUE H Y, YAO C W, et al. Sparse coding guided spatiotemporal feature learning for abnormal event detection in large videos[J]. IEEE Transactions on Multimedia, 2019, 21(1):246-255.
[65] 胡正平, 张乐, 尹艳华. 时空深度特征AP聚类的稀疏表示视频异常检测算法[J]. 信号处理, 2019, 35(3):386-395. HU Z P, ZHANG L, YIN Y H. Video anomaly detection by AP clustering sparse representation based on spatial-temporal deep feature model[J]. Journal of Signal Processing, 2019, 35(3):386-395. (in Chinese)
[66] FENG J, ZHANG C, HAO P W. Online learning with self-organizing maps for anomaly detection in crowd scenes[C]//Proceedings of the 2010 20th International Conference on Pattern Recognition. Istanbul, Turkey, 2010:3599-3602.
[67] SUN Q R, LIU H, HARADA T. Online growing neural gas for anomaly detection in changing surveillance scenes[J]. Pattern Recognition, 2017, 64:187-201.
[68] FAN Y X, WEN G J, LI D R, et al. Video anomaly detection and localization via Gaussian mixture fully convolutional variational autoencoder[Z/OL]. arXiv:1805.11223, 2018.
[69] CHALAPATHY R, MENON A K, CHAWLA S. Anomaly detection using one-class neural networks[Z/OL]. arXiv:1802.06360, 2018.
[70] HASAN M, CHOI J, NEUMANN J, et al. Learning temporal regularity in video sequences[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA, 2016:733-742.
[71] SABOKROU M, FATHY M, HOSEINI M. Video anomaly detection and localisation based on the sparsity and reconstruction error of auto-encoder[J]. Electronics Letters, 2016, 52(13):1122-1124.
[72] MUNAWAR A, VINAYAVEKHIN P, DE MAGISTRIS G. Limiting the reconstruction capability of generative neural network using negative learning[C]//Proceedings of the 2017 IEEE 27th International Workshop on Machine Learning for Signal Processing. Tokyo, Japan, 2017:1-6.
[73] CHALAPATHY R, MENON A K, CHAWLA S. Robust, deep and inductive anomaly detection[C]//Proceedings of 2017 Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Skopje, Macedonia, 2017:36-51.
[74] NOGAS J, KHAN S S, MIHAILIDIS A. DeepFall:Non-invasive fall detection with deep spatio-temporal convolutional autoencoders[Z/OL]. arXiv:1809.00977, 2018.
[75] ZHAO Y R, DENG B, SHEN C, et al. Spatio-temporal autoencoder for video anomaly detection[C]//Proceedings of the 25th ACM International Conference on Multimedia. Mountain View, USA, 2017:1933-1941.
[76] WANG T, QIAO M N, LIN Z W, et al. Generative neural networks for anomaly detection in crowded scenes[J]. IEEE Transactions on Information Forensics and Security, 2019, 14(5):1390-1399.
[77] RAVANBAKHSH M, NABI M, SANGINETO E, et al. Abnormal event detection in videos using generative adversarial nets[C]//Proceedings of 2017 IEEE International Conference on Image Processing. Beijing, China, 2017:1577-1581.
[78] RAVANBAKHSH M, SANGINETO E, NABI M, et al. Training adversarial discriminators for cross-channel abnormal event detection in crowds[C]//Proceedings of 2019 IEEE Winter Conference on Applications of Computer Vision. Waikoloa Village, USA, 2019:1896-1904.
[79] SABOKROU M, KHALOOEI M, FATHY M, et al. Adversarially learned one-class classifier for novelty detection[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA, 2018:3379-3388.
[80] SABOKROU M, POURREZA M, FAYYAZ M, et al. AVID:Adversarial visual irregularity detection[Z/OL]. arXiv:1805.09521, 2018.
[81] ZENATI H, FOO C S, LECOUAT B, et al. Efficient GAN-based anomaly detection[Z/OL]. arXiv:1802.06222, 2018.
[82] AKCAY S, ATAPOUR-ABARGHOUEI A, BRECKON T P. GANomaly:Semi-supervised anomaly detection via adversarial training[Z/OL]. arXiv:1805.06725, 2018.
[83] SCHLEGL T, SEEBÖCK P, WALDSTEIN S M, et al. Unsupervised anomaly detection with generative adversarial networks to guide marker discovery[C]//Proceedings of the 25th International Conference on Information Processing in Medical Imaging. Boone, USA, 2017:146-147.
[84] VU H, NGUYEN T D, TRAVERS A, et al. Energy-based localized anomaly detection in video surveillance[C]//Proceedings of the 21st Pacific-Asia Conference on Knowledge Discovery and Data Mining. Jeju, South Korea, 2017:641-653.
[85] VU H, NGUYEN T D, LE T, et al. Robust anomaly detection in videos using multilevel representations[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2019, 33(1):5216-5223.
[86] DONAHUE J, KRÄHENBÜHL P, DARRELL T. Adversarial feature learning[Z/OL]. arXiv:1605.09782, 2016.
[87] VILLEGAS R, YANG J M, HONG S, et al. Decomposing motion and content for natural video sequence prediction[Z/OL]. arXiv:1706.08033, 2017.
[88] LEE S, KIM H G, RO Y M. STAN:Spatio-temporal adversarial networks for abnormal event detection[C]//Proceedings of 2018 IEEE International Conference on Acoustics, Speech and Signal Processing. Calgary, Canada, 2018:1323-1327.
[89] D'AVINO D, COZZOLINO D, POGGI G, et al. Autoencoder with recurrent neural networks for video forgery detection[J]. Electronic Imaging, 2017, 2017(7):92-99.
[90] CHONG Y S, TAY Y H. Abnormal event detection in videos using spatiotemporal autoencoder[C]//Proceedings of the 14th International Symposium on Neural Networks. Hokkaido, Japan, 2017:189-196.
[91] MEDEL J R, SAVAKIS A. Anomaly detection in video using predictive convolutional long short-term memory networks[Z/OL]. arXiv:1612.00390, 2016.
[92] LUO W X, LIU W, GAO S H. Remembering history with convolutional LSTM for anomaly detection[C]//Proceedings of 2017 IEEE International Conference on Multimedia and Expo. Hong Kong, China, 2017:439-444.
[93] SULTANI W, CHEN C, SHAH M. Real-world anomaly detection in surveillance videos[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA, 2018:6479-6488.
[94] ADAM A, RIVLIN E, SHIMSHONI I, et al. Robust real-time unusual event detection using multiple fixed-location monitors[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2008, 30(3):555-560.
[95] LI N N, WU X Y, XU D, et al. Spatio-temporal context analysis within video volumes for anomalous-event detection and localization[J]. Neurocomputing, 2015, 155:309-319.
[96] RAVANBAKHSH M, NABI M, MOUSAVI H, et al. Plug-and-play CNN for crowd motion analysis:An application in abnormal event detection[C]//Proceedings of 2018 IEEE Winter Conference on Applications of Computer Vision. Lake Tahoe, USA, 2018:1689-1698. |
|
Viewed |
|
|
|
Full text
|
|
|
|
|
Abstract
|
|
|
|
|
Cited |
|
|
|
|
| |
Shared |
|
|
|
|
| |
Discussed |
|
|
|
|
2020,
Vol. 60
