Road segmentation using full convolutional neural networks with conditional random fields

doi:10.16511/j.cnki.qhdxxb.2018.21.013

Download: PDF(2135 KB)
Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks

Abstract Common road segmentation methods are often limited by environmental noise and the roughness of the segmenting edges. A road segmentation method was developed to address these shortcomings by combining a fully convolutional neural network and a conditional random field. The feature representation in the neural networks models the road segmentation as a binary classification problem. A VGG-16 deep convolutional neural network based fully convolutional network was constructed to classify each road image end to end into the road and the background. Then, the fully-connected conditional random field (CRF) was used for fine segmentation to refine the coarse edges obtained from the binary classification. Tests of road segmentation benchmark datasets acquired in real environments show that this method can achieve 98.13% segmentation accuracy and real-time processing with 0.84 s perimage.

Keywords image pattern recognition road segmentation full convolutional neural network conditional random field

Issue Date: 15 August 2018

	Service

	E-mail this article
	E-mail Alert
	RSS
	Articles by authors

	SONG Qingsong
	ZHANG Chao
	CHEN Yu
	WANG Xingli
	YANG Xiaojun

Cite this article:

SONG Qingsong,ZHANG Chao,CHEN Yu, et al. Road segmentation using full convolutional neural networks with conditional random fields[J]. Journal of Tsinghua University(Science and Technology), 2018, 58(8): 725-731.

URL:

http://jst.tsinghuajournals.com/EN/10.16511/j.cnki.qhdxxb.2018.21.013 OR http://jst.tsinghuajournals.com/EN/Y2018/V58/I8/725

[1] ALVAREZ J M, GEVERS T, LOPEZ A M. Road detection by one-class color classification:Dataset and experiments[R/OL]. (2014-12-18)[2017-10-20]. https://arxiv.org/abs/1412.3506.
[2] GRAOVAC S, GOMA A. Detection of road image borders based on texture classification[J]. International Journal of Advanced Robotic Systems, 2012, 9(242):1-12.
[3] WANG Y, SHEN D G, TEOH E K. Lane detection using spline model[J]. Pattern Recognition Letters, 2000, 21(8):677-689.
[4] WANG Y, TEOH E K, SHEN D G. Lane detection and tracking using B-Snake[J]. Image and Vision Computing, 2004, 22(4):269-280.
[5] LECUN Y, BOTTOU L, BENGIO Y, et al. Gradient-based learning applied to document recognition[J]. Proceedings of the IEEE, 1998, 86(11):2278-2324.
[6] NING F, DELHOMME D, LECUN Y, et al. Toward automatic phenotyping of developing embryos from videos[J]. IEEE Transactions on Image Processing, 2005, 14(9):1360-1371.
[7] GANIN Y, LEMPITSKY V. N4-fields:Neural network nearest neighbor fields for image transforms[C]//Proceedings of the 12th Asian Conference on Computer Vision. Singapore:Springer, 2014:536-551.
[8] LONG J, SHELHAMER E, DARRELL T. Fully convolutional networks for semantic segmentation[C]//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston, USA:IEEE, 2015:3431-3440.
[9] LAFFERTY J D, MCCALLUM A, PEREIRA F C N. Conditional random fields:Probabilistic models for segmenting and labeling sequence data[C]//Proceedings of the 18th International Conference on Machine Learning. San Francisco, USA:Morgan Kaufmann Publishers Inc., 2001:282-289.
[10] SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[C]//2015 International Conference on Learning Representations. Lille, France:University of Oxford, 2015:1-14.
[11] SZEGEDY C, LIU W, JIA Y Q, et al. Going deeper with convolutions[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston, USA:IEEE, 2015:1-9.
[12] HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA:IEEE, 2016:770-778.
[13] ZEILER M D, KRISHNAN D, TAYLOR G W, et al. Deconvolutional networks[C]//2010 IEEE Conference on Computer Vision and Pattern Recognition. San Francisco, CA, USA:IEEE, 2010, 238(6):2528-2535.
[14] ZEILER M D, FERGUS R. Visualizing and understanding convolutional networks[C]//The 13th European Conference on Computer Vision. Zurich, Switzerland:Springer, 2014:818-833.
[15] SHI W Z, CABALLERO J, HUSZÁR F, et al. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, Nevada, USA:IEEE, 2016:1874-1883.
[16] Avisynth wiki. Resampling[R/OL]. (2016-09-22)[2017-11-12]. http://avisynth.nl/index.php/Resampling.
[17] KRAHENBUHL P, KOLTUN V. Efficient inference in fully connected crfs with Gaussian edge potentials[C]//The 24th International Conference on Neural Information Processing Systems. Granada, Spain:Neural Information Processing Systems Foundation, Inc., 2011:109-117.
[18] ZHENG S, JAYASUMANA S, ROMERA-PAREDES B, et al. Conditional random fields as recurrent neural networks[C]//2015 IEEE International Conference on Computer Vision, Santiago, Chile:IEEE, 2015:1529-1537.
[19] ÁLVAREZ J M, LÓPEZ A M, GEVERS T, et al. Combining priors, appearance, and context for road detection[J]. IEEE Transactions on Intelligent Transportation Systems, 2014, 15(3):1168-1178.
[20] The PASCAL visual object classes challenge 2012(VOC2012)[R/OL]. (2012-02-01)[2017-09-01]. http://www.pascal-network.org/challenges/VOC/voc2012/workshop/index.html.
[21] KINGMA D P, Ba J. Adam:A method for stochastic optimization[R/OL]. (2017-01-30)[2017-08-21]. https://arxiv.org/abs/1412.6980.

[1]	CHEN Chuangang, HU Jinqiu, HAN Zicong, CHEN Yiyue, XIAO Shangrui. Knowledge graph based early warning method for accident evolution in natural gas pipeline station abroad for harsh environmental conditions[J]. Journal of Tsinghua University(Science and Technology), 2022, 62(6): 1081-1087.
[2]	QIAN Yili, ZHANG Ermeng. Identification of prosodic phrases based on syntax dependency and conditional random fields[J]. Journal of Tsinghua University(Science and Technology), 2019, 59(7): 530-536.
[3]	LI Xu, TU Ming, WU Chao, GUO Yanmeng, NA Yueyue, FU Qiang, YAN Yonghong. Single-channel speech separation with non-negative matrix factorization and factorial conditional random fields[J]. Journal of Tsinghua University(Science and Technology), 2017, 57(1): 84-88.
[4]	LIU Zewen, DING Dong, LI Chunwen. Chinese word segmentation method for short Chinese text based on conditional random fields[J]. Journal of Tsinghua University(Science and Technology), 2015, 55(8): 906-910,915.