基于深度信念卷积神经网络的图像识别与分类

刘琼, 李宗贤, 孙富春, 田永鸿, 曾炜

清华大学学报（自然科学版） ›› 2018, Vol. 58 ›› Issue (9) : 781-787.

PDF(2527 KB)
横山亮次奖  |  百年刊庆  |  中文  |  English
PDF(2527 KB)
清华大学学报（自然科学版） ›› 2018, Vol. 58 ›› Issue (9) : 781-787. DOI: 10.16511/j.cnki.qhdxxb.2018.22.034
自动化

基于深度信念卷积神经网络的图像识别与分类

  • 刘琼1, 李宗贤2, 孙富春3, 田永鸿2, 曾炜2
作者信息 +

Image recognition and classification by deep belief-convolutional neural networks

  • LIU Qiong1, LI Zongxian2, SUN Fuchun3, TIAN Yonghong2, ZENG Wei2
Author information +
文章历史 +

摘要

针对基于卷积神经网络的图像识别采用随机初始化网络权值的方法易收敛到局部最优值的问题，该文提出了一种结合无监督和有监督学习的网络权值预训练算法。融合零成分分析白化与深度信念网络预学习得到的特征，对卷积神经网络权值进行初始化；通过卷积、池化等操作，对训练样本进行特征提取并使用全连接网络对特征进行分类；计算分类损失函数并优化网络参数。在公开图像数据库中进行了大量实验，与公开最佳算法比较，该算法在MNIST中的识别错误率降低了0.1%，在Caltech101中的分类准确率提升了0.56%，验证了该算法优于现有算法。

Abstract

Convolutional neural network (CNN) would easily converge to the local minimum if the network was randomly initialized in image classification tasks. A deep belief network pre-training method was developed by merging unsupervised and supervised methods. Feature sets were extracted from the image patches of zero component analysis (ZCA) whitening and deep belief pre-training to initialize weights of CNNs. Then, convolution features were extracted from the training samples by applying convolution and pooling operations and classified to a specific category through a fully connected network. Finally, the loss value was computed for global optimization. Extensive experimental evaluations on some public datasets show that this method is simple but very effective with the error rate decrease of 0.1% on MNIST and the accuracy increase of 0.56% on Caltech101, which indicates that this method is superior to similar methods.

关键词

深度信念网络 / 图像识别 / 卷积神经网络

Key words

deep belief networks / image recognition / convolutional neural networks

引用本文

EndNote

Ris (Procite)

Bibtex

导出引用
刘琼, 李宗贤, 孙富春, 田永鸿, 曾炜. 基于深度信念卷积神经网络的图像识别与分类[J]. 清华大学学报（自然科学版）. 2018, 58(9): 781-787 https://doi.org/10.16511/j.cnki.qhdxxb.2018.22.034
LIU Qiong, LI Zongxian, SUN Fuchun, TIAN Yonghong, ZENG Wei. Image recognition and classification by deep belief-convolutional neural networks[J]. Journal of Tsinghua University(Science and Technology). 2018, 58(9): 781-787 https://doi.org/10.16511/j.cnki.qhdxxb.2018.22.034

参考文献

[1] YUILLE A L, HALLINAN P W, COHEN D S. Feature extraction from faces using deformable templates[J]. International Journal of Computer Vision, 1992, 8(2):99-111.
[2] OJALA T, PIETIKAINEN M, MAENPAA T. Multiresolution gray-scale and rotation invariant texture classification with local binary patterns[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2002, 24(7):971-987.
[3] DALAL N, TRIGGS B. Histograms of oriented gradients for human detection[C]//Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. San Diego, USA, 2005.
[4] LOWE D G. Object recognition from local scale-invariant features[C]//Proceedings of the Seventh IEEE International Conference on Computer Vision. Kerkyra, Greece, 1999.
[5] HINTON G E. Learning multiple layers of representation[J]. Trends in Cognitive Sciences, 2007, 11(10):428-434.
[6] HINTON G E, SALAKHUTDINOV R R. Reducing the dimensionality of data with neural networks[J]. Science, 2006, 313(5786):504-507.
[7] SIHAG S, DUTTA P K. Faster method for deep belief network based object classification using DWT[J]. arXiv preprint arXiv:1511.06276, 2015.
[8] TORRES-CARRASQUILLO P A, SINGER E, KOHLER M A, et al. Approaches to language identification using Gaussian mixture models and shifted delta cepstral features[C]//Proceedings of the 7th International Conference on Spoken Language Processing. Denver, USA, 2002.
[9] COLLOBERT R, BENGIO S. SVMTorch:Support vector machines for large-scale regression problems[J]. Journal of Machine Learning Research, 2000, 1(2):143-160.
[10] GOODFELLOW I J, WARDE-FARLEY D, MIRZA M, et al. Maxout networks[J]. arXiv preprint arXiv:1302.4389, 2013.
[11] JARRETT K, KAVUKCUOGLU K, RANZATO M, et al. What is the best multi-stage architecture for object recognition?[C]//IEEE 12th International Conference on Computer Vision. Kyoto, Japan, 2009:2146-2153.
[12] HINTON G E. A practical guide to training restricted Boltzmann machines[M]//MONTAVON G, ORR G B, MVLLER K R. Neural networks:Tricks of the trade. 2nd ed. Berlin, Germany:Springer, 2012.
[13] KAVUKCUOGLU K, SERMANET P, BOUREAU Y L, et al. Learning convolutional feature hierarchies for visual recognition[C]//Proceedings of the 23rd International Conference on Neural Information Processing Systems. Vancouver, Canada, 2010:1090-1098.
[14] LEE H, GROSSE R, RANGANATH R, et al. Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations[C]//Proceedings of the 26th Annual International Conference on Machine Learning. Montreal, Canada, 2009:609-616.
[15] DONAHUE J, JIA Y Q, VINYALS O, et al. DeCAF:A deep convolutional activation feature for generic visual recognition[C]//Proceedings of the 31st International Conference on Machine Learning. Beijing, 2014:647-655.
[16] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks[C]//Proceedings of the 25th International Conference on Neural Information Processing Systems. Lake Tahoe, USA, 2012:1097-1105.
[17] SIMONYAN K, ZISSERMAN A. Two-stream convolutional networks for action recognition in videos[C]//Proceedings of the 27th International Conference on Neural Information Processing Systems. Montréal, Canada, 2014:568-576.
[18] COLLOBERT R, WESTON J, BOTTOU L, et al. Natural language processing (almost) from scratch[J]. The Journal of Machine Learning Research, 2011, 12:2493-2537.
[19] ZEILER M D, FERGUS R. Stochastic pooling for regularization of deep convolutional neural networks[J]. arXiv preprint arXiv:1301.3557, 2013.
[20] YU K, LIN Y Q, LAFFERTY J. Learning image representations from the pixel level via hierarchical sparse coding[C]//IEEE International Conference on Computer Vision and Pattern Recognition. Colorado Springs, USA, 2011:1713-1720.
[21] BRUNA J, MALLAT S. Invariant scattering convolution networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(8):1872-1886.
[22] CHAN T H, JIA K, GAO S H, et al. PCANet:A simple deep learning baseline for image classification?[J]. IEEE Transactions on Image Processing, 2015, 24(12):5017-5032.
[23] SERRE T, KREIMAN G, KOUH M, et al. A quantitative theory of immediate visual recognition[J]. Progress in Brain Research, 2007, 165:33-56.
[24] COATES A, NG A Y. Selecting receptive fields in deep networks[C]//Proceedings of the 24th International Conference on Neural Information Processing Systems. Granada, Spain, 2011:2528-2536.
[25] DENG Li. The MNIST database of handwritten digit images for machine learning research[J]. IEEE Signal Processing Magazine, 2012, 29(6):141-142.
[26] BELONGIE S, MALIK J, PUZICHA J. Shape matching and object recognition using shape contexts[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2002, 24(4):509-522.
[27] HE K M, ZHANG X Y, REN S Q, et al. Spatial pyramid pooling in deep convolutional networks for visual recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(9):1904-1916.
[28] CIREŞAN D, MEIER U, MASCI J, et al. Multi-column deep neural network for traffic sign classification[J]. Neural Networks, 2012, 32:333-338.
[29] STALLKAMP J, SCHLIPSING M, SALMEN J, et al. Man vs. computer:Benchmarking machine learning algorithms for traffic sign recognition[J]. Neural Networks, 2012, 32:323-332.
[30] SERMANET P, LECUN Y. Traffic sign recognition with multi-scale convolutional networks[C]//Proceedings of 2011 International Joint Conference on Neural Networks. San Jose, USA, 2011:2809-2813.
[31] ZAKLOUTA F, STANCIULESCU B, HAMDOUN O. Traffic sign classification using K-d trees and random forests[C]//Proceedings of 2011 International Joint Conference on Neural Networks. San Jose, USA, 2011.

基金

国家自然科学基金项目（61327809，91420302，61633002）；国家“九七三”重点基础研究发展计划（2015CB351806）；2018年度北京市属高校青年拔尖人才项目（CIT&TCD201804054）

PDF(2527 KB)

Accesses

Citation

Detail
段落导航
相关文章
扫码关注，本刊微信公众号
地址：清华大学东门外学研大厦B座908室　
邮编：100084
电话：010-62788108/62771385/62792976/62792994　
E-mail：xuebaost@tsinghua.edu.cn
访问统计
    总访问量
    今日访问
    在线人数
　
版权所有 © 《清华大学学报（自然科学版）》编辑部

/

〈 〉