1. Information Technology Center, Tsinghua University, Beijing 100084, China; 2. State Key Laboratory of Automotive Safety and Energy, Tsinghua University, Beijing 100084, China; 3. Department of Computer Science and Technology, Tsinghua University, Beijing 100084, China
Abstract：This paper introduces target recognition and detection methods based on the convolutional neural network (CNN) model, the improved regions with convolutional neural network (R-CNN) and the task-assistant convolutional neural network (TA-CNN) model for pedestrian detection. This paper also describes stereo matching based on a deep learning model for stereo matching using the Siamese network. Multi-source data fusion is also introduced based on a vision sensor, a radar sensor and a camera using a deep learning network. The CNN is used for end-to-end horizontal and vertical control of autonomous vehicles. Deep learning is widely used in the perception level, decision-making level and control level in automatic driving systems to continuously improve the perception, detection, decision-making and control accuracy. Analyses show that deep learning will improve of autonomous driving systems.
[1] DÖRR D, GRABENGIESSER D, GAUTERIN F. Online driving style recognition using fuzzy logic[C]//Proceedings of the 17th International Conference on Intelligent Transportation Systems (ITSC). Qingdao, China:IEEE, 2014:1021-1026. [2] DALAL N, TRIGGS B. Histograms of oriented gradients for human detection[C]//Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. San Diego, CA, USA:IEEE, 2005, 1:886-893. [3] WU B, NEVATIA R. Detection of multiple, partially occluded humans in a single image by Bayesian combination of edgelet part detectors[C]//Proceedings of the 10th IEEE International Conference on Computer Vision. Beijing, China:IEEE, 2005, 1:90-97. [4] GAVRILA D M. A Bayesian, exemplar-based approach to hierarchical shape matching[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2007, 29(8):1408-1421. [5] MU Y D, YAN S C, LIU Y, et al. Discriminative local binary patterns for human detection in personal album[C]//Proceedings of the 2008 IEEE Conference on Computer Vision and Pattern Recognition. Anchorage, Alaska, USA:IEEE, 2008:1-8. [6] WANG X Y, HAN T X, YAN S C. An HOG-LBP human detector with partial occlusion handling[C]//Proceedings of the 12th International Conference on Computer Vision. Kyoto, Japan:IEEE, 2009:32-39. [7] TUZEL O, PORIKLI F, MEER P. Pedestrian detection via classification on Riemannian manifolds[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2008, 30(10):1713-1727. [8] WATANABE T, ITO S, YOKOI K. Co-occurrence histograms of oriented gradients for human detection[J]. IPSJ Transactions on Computer Vision and Applications, 2010, 2:39-47. [9] DOLLAR P, TU Z W, PERONA P, et al. Integral channel features[C]//Proceedings of the British Machine Vision Conference. London, UK:BMVC, 2009:1-11. [10] GAO W, AI H Z, LAO S H. Adaptive Contour Features in oriented granular space for human detection and segmentation[C]//Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition. Miami, FL, USA:IEEE, 2009:1786-1793. [11] LIU Y Z, SHAN S G, ZHANG W C, et al. Granularity-tunable gradients partition (GGP) descriptors for human detection[C]//Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition. Miami, FL, USA:IEEE, 2009:1255-1262. [12] WU J X, GEYER C, REHG J M. Real-time human detection using contour cues[C]//Proceedings of the 2011 IEEE International Conference on Robotics and Automation. Shanghai, China:IEEE, 2011:860-867. [13] WALK S, MAJER N, SCHINDLER K, et al. New features and insights for pedestrian detection[C]//Proceedings of the 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. San Francisco, CA, USA:IEEE, 2010:1030-1037. [14] PLÖCHL M, EDELMANN J. Driver models in automobile dynamics application[J]. Vehicle System Dynamics, 2007, 45(7-8):699-741. [15] GIRSHICK R, DONAHUE J, DARRELL T, et al. Rich feature hierarchies for accurate object detection and semantic segmentation[C]//Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition. Columbus, OH, USA:IEEE, 2014:580-587. [16] WALLACE B, GOUBRAN R, KNOEFEL F, et al. Measuring variation in driving habits between drivers[C]//Proceedings of the 2014 IEEE International Symposium on Medical Measurements and Applications. Lisboa, Portugal:IEEE, 2014:1-6. [17] GIRSHICK R. Fast R-CNN[C]//Proceedings of the 2015 IEEE International conference on Computer Vision. Santiago, Chile:IEEE, 2015:1440-1448. [18] REN S Q, HE K M, GIRSHICK R, et al. Faster R-CNN:Towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6):1137-1149. [19] TIAN Y L, LUO P, WANG X G, et al. Pedestrian detection aided by deep learning semantic tasks[C]//Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston, MA, USA:IEEE, 2015:5079-5087. [20] LUO L H. Adaptive cruise control design with consideration of humans' driving psychology[C]//Proceedings of the 11th World Congress on Intelligent Control and Automation. Shenyang, China:IEEE, 2014:2973-2978. [21] PACHECO J E, LÓPEZ E. Monitoring driving habits through an automotive CAN network[C]//Proceedings of the 23rd International Conference on Electronics, Communications and Computing. Cholula, Mexico:IEEE, 2013:138-143. [22] MVHLMANN K, MAIER D, HESSER J, et al. Calculating dense disparity maps from color stereo images, an efficient implementation[J]. International Journal of Computer Vision, 2002, 47(1-3):79-88. [23] CHOPRA S, HADSELL R, LECUN Y. Learning a similarity metric discriminatively, with application to face verification[C]//Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. San Diego, CA, USA:IEEE, 2005, 1:539-546. [24] LUO W J, SCHWING A G, URTASUN R. Efficient deep learning for stereo matching[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV, USA:IEEE, 2016:5695-5703. [25] BUTAKOV V, IOANNOU P. Driving autopilot with personalization feature for improved safety and comfort[C]//Proceedings of the 18th International Conference on Intelligent Transportation Systems. Las Palmas, Spain:IEEE, 2015:387-393. [26] ALJAAFREH A, ALSHABATAT N, NAJIM AL-DIN M S. Driving style recognition using fuzzy logic[C]//Proceedings of the 2012 IEEE International Conferenceon Vehicular Electronics and Safety. Istanbul, Turkey:IEEE, 2012:460-463. [27] JOHNSON D A, TRIVEDI M M. Driving style recognition using a smartphone as a sensor platform[C]//Proceedings of the 14th International IEEE Conference on Intelligent Transportation Systems. Washington, DC, USA:IEEE, 2011:1609-1615. [28] SCHLOSSER J, CHOW C K, KIRA Z. Fusing LIDAR and images for pedestrian detection using convolutional neural networks[C]//Proceedings of the2016 IEEE International Conference on Robotics and Automation. Stockholm, Sweden:IEEE, 2016:2198-2205. [29] VAN LY M, MARTIN S, TRIVEDI M M. Driver classification and driving style recognition using inertial sensors[C]//Proceedings of the 2013 IEEE Intelligent Vehicles Symposium (IV). Gold Coast, QLD, Australia:IEEE, 2013:1040-1045. [30] POMERLEAU D A. ALVINN:An autonomous land vehicle in a neural network[C]//Advances in Neural Information Processing Systems. San Francisco, CA, USA:ACM, 1989:305-313. [31] BOJARSKI M, DEL TESTA D, DWORAKOWSKI D, et al. End to end learning for self-driving cars[J/OL]. (2016-04-25). http://arxiv.org/pdf/1604.07316.pdf. [32] CHEN C Y, SEFF A, KORNHAUSER A, et al. DeepDriving:Learning affordance for direct perception in autonomous driving[C]//Proceedings of the 2015 IEEE International Conference on Computer Vision. Santiago, Chile:IEEE, 2015:2722-2730.
2018, Vol. 58 
