Image recognition and classification by deep belief-convolutional neural networks
LIU Qiong1, LI Zongxian2, SUN Fuchun3, TIAN Yonghong2, ZENG Wei2
1. School of Automation, Beijing Information Science and Technology University, Beijing 100192, China; 2. National Engineering Laboratory for Video Technology, School of Electronics Engineering and Computer Science, Peking University, Beijing 100871, China; 3. State Key Laboratory of Intelligence Technology and System, Department of Computer Science, Tsinghua University, Beijing 100084, China
Abstract：Convolutional neural network (CNN) would easily converge to the local minimum if the network was randomly initialized in image classification tasks. A deep belief network pre-training method was developed by merging unsupervised and supervised methods. Feature sets were extracted from the image patches of zero component analysis (ZCA) whitening and deep belief pre-training to initialize weights of CNNs. Then, convolution features were extracted from the training samples by applying convolution and pooling operations and classified to a specific category through a fully connected network. Finally, the loss value was computed for global optimization. Extensive experimental evaluations on some public datasets show that this method is simple but very effective with the error rate decrease of 0.1% on MNIST and the accuracy increase of 0.56% on Caltech101, which indicates that this method is superior to similar methods.
刘琼, 李宗贤, 孙富春, 田永鸿, 曾炜. 基于深度信念卷积神经网络的图像识别与分类[J]. 清华大学学报（自然科学版）, 2018, 58(9): 781-787.
LIU Qiong, LI Zongxian, SUN Fuchun, TIAN Yonghong, ZENG Wei. Image recognition and classification by deep belief-convolutional neural networks. Journal of Tsinghua University(Science and Technology), 2018, 58(9): 781-787.
[1] YUILLE A L, HALLINAN P W, COHEN D S. Feature extraction from faces using deformable templates[J]. International Journal of Computer Vision, 1992, 8(2):99-111. [2] OJALA T, PIETIKAINEN M, MAENPAA T. Multiresolution gray-scale and rotation invariant texture classification with local binary patterns[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2002, 24(7):971-987. [3] DALAL N, TRIGGS B. Histograms of oriented gradients for human detection[C]//Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. San Diego, USA, 2005. [4] LOWE D G. Object recognition from local scale-invariant features[C]//Proceedings of the Seventh IEEE International Conference on Computer Vision. Kerkyra, Greece, 1999. [5] HINTON G E. Learning multiple layers of representation[J]. Trends in Cognitive Sciences, 2007, 11(10):428-434. [6] HINTON G E, SALAKHUTDINOV R R. Reducing the dimensionality of data with neural networks[J]. Science, 2006, 313(5786):504-507. [7] SIHAG S, DUTTA P K. Faster method for deep belief network based object classification using DWT[J]. arXiv preprint arXiv:1511.06276, 2015. [8] TORRES-CARRASQUILLO P A, SINGER E, KOHLER M A, et al. Approaches to language identification using Gaussian mixture models and shifted delta cepstral features[C]//Proceedings of the 7th International Conference on Spoken Language Processing. Denver, USA, 2002. [9] COLLOBERT R, BENGIO S. SVMTorch:Support vector machines for large-scale regression problems[J]. Journal of Machine Learning Research, 2000, 1(2):143-160. [10] GOODFELLOW I J, WARDE-FARLEY D, MIRZA M, et al. Maxout networks[J]. arXiv preprint arXiv:1302.4389, 2013. [11] JARRETT K, KAVUKCUOGLU K, RANZATO M, et al. What is the best multi-stage architecture for object recognition?[C]//IEEE 12th International Conference on Computer Vision. Kyoto, Japan, 2009:2146-2153. [12] HINTON G E. A practical guide to training restricted Boltzmann machines[M]//MONTAVON G, ORR G B, MVLLER K R. Neural networks:Tricks of the trade. 2nd ed. Berlin, Germany:Springer, 2012. [13] KAVUKCUOGLU K, SERMANET P, BOUREAU Y L, et al. Learning convolutional feature hierarchies for visual recognition[C]//Proceedings of the 23rd International Conference on Neural Information Processing Systems. Vancouver, Canada, 2010:1090-1098. [14] LEE H, GROSSE R, RANGANATH R, et al. Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations[C]//Proceedings of the 26th Annual International Conference on Machine Learning. Montreal, Canada, 2009:609-616. [15] DONAHUE J, JIA Y Q, VINYALS O, et al. DeCAF:A deep convolutional activation feature for generic visual recognition[C]//Proceedings of the 31st International Conference on Machine Learning. Beijing, 2014:647-655. [16] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks[C]//Proceedings of the 25th International Conference on Neural Information Processing Systems. Lake Tahoe, USA, 2012:1097-1105. [17] SIMONYAN K, ZISSERMAN A. Two-stream convolutional networks for action recognition in videos[C]//Proceedings of the 27th International Conference on Neural Information Processing Systems. Montréal, Canada, 2014:568-576. [18] COLLOBERT R, WESTON J, BOTTOU L, et al. Natural language processing (almost) from scratch[J]. The Journal of Machine Learning Research, 2011, 12:2493-2537. [19] ZEILER M D, FERGUS R. Stochastic pooling for regularization of deep convolutional neural networks[J]. arXiv preprint arXiv:1301.3557, 2013. [20] YU K, LIN Y Q, LAFFERTY J. Learning image representations from the pixel level via hierarchical sparse coding[C]//IEEE International Conference on Computer Vision and Pattern Recognition. Colorado Springs, USA, 2011:1713-1720. [21] BRUNA J, MALLAT S. Invariant scattering convolution networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(8):1872-1886. [22] CHAN T H, JIA K, GAO S H, et al. PCANet:A simple deep learning baseline for image classification?[J]. IEEE Transactions on Image Processing, 2015, 24(12):5017-5032. [23] SERRE T, KREIMAN G, KOUH M, et al. A quantitative theory of immediate visual recognition[J]. Progress in Brain Research, 2007, 165:33-56. [24] COATES A, NG A Y. Selecting receptive fields in deep networks[C]//Proceedings of the 24th International Conference on Neural Information Processing Systems. Granada, Spain, 2011:2528-2536. [25] DENG Li. The MNIST database of handwritten digit images for machine learning research[J]. IEEE Signal Processing Magazine, 2012, 29(6):141-142. [26] BELONGIE S, MALIK J, PUZICHA J. Shape matching and object recognition using shape contexts[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2002, 24(4):509-522. [27] HE K M, ZHANG X Y, REN S Q, et al. Spatial pyramid pooling in deep convolutional networks for visual recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(9):1904-1916. [28] CIREŞAN D, MEIER U, MASCI J, et al. Multi-column deep neural network for traffic sign classification[J]. Neural Networks, 2012, 32:333-338. [29] STALLKAMP J, SCHLIPSING M, SALMEN J, et al. Man vs. computer:Benchmarking machine learning algorithms for traffic sign recognition[J]. Neural Networks, 2012, 32:323-332. [30] SERMANET P, LECUN Y. Traffic sign recognition with multi-scale convolutional networks[C]//Proceedings of 2011 International Joint Conference on Neural Networks. San Jose, USA, 2011:2809-2813. [31] ZAKLOUTA F, STANCIULESCU B, HAMDOUN O. Traffic sign classification using K-d trees and random forests[C]//Proceedings of 2011 International Joint Conference on Neural Networks. San Jose, USA, 2011.
2018, Vol. 58 
