PDF(3413 KB)
Overview of research progress on 3-D face recognition
Changwei LUO,Jun YU,Lingyun YU,Yali LI,Shengjin WANG
Journal of Tsinghua University(Science and Technology) ›› 2021, Vol. 61 ›› Issue (1) : 77-88.
PDF(3413 KB)
PDF(3413 KB)
Overview of research progress on 3-D face recognition
Research on 3-D face recognition has made great progress in recent years. 3-D face recognition is more effective than 2-D face recognition. Its main feature is the use of 3-D shape data for recognition. The 3-D face recognition methods are categorized into three types based on the source of the 3-D shape data with methods based on 2-D color images, high quality 3-D scanning data, and low quality RGB-D images. This study reviews these methods and discusses their advantages and disadvantages. This paper then reviews the use of deep learning methods for 3-D face recognition. Besides, 3-D and 2-D face data fusion methods are reviewed for bi-modal face recognition. The commonly used 3-D face databases are also summarized. Finally, the main difficulties and future development trends of 3-D face recognition are discussed.
| 1 | WANG M, DENG W. Deep face recognition: A survey[Z/OL]. (2018-04-18)[2020-07-01]. https://arxiv.org/abs/1804.06655. |
| 3 | PATIL H , KOTHARI A , BHURCHANDI K . 3-D face recognition:Features, databases, algorithms and challenges[J]. Artificial Intelligence Review, 2015. 44 (3): 393- 441. |
| 4 | SOLTANPOUR S , BOUFAMA B , WU Q M . A survey of local feature methods for 3D face recognition[J]. Pattern Recognition, 2017. 72, 391- 406. |
| 5 | BLANZ V , VETTER T . Face recognition based on fitting a 3D morphable model[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2003. 25 (9): 1063- 1074. |
| 7 | PAYSAN P, KNOTHE R, AMBERG B, et al. A 3D face model for pose and illumination invariant face recognition[C]//20096th IEEE International Conference on Advanced Video and Signal Based Surveillance. Genova, Italy: IEEE, 2009: 296-301. |
| 8 | LIU F , ZHAO Q J , LIU X M , et al. Joint face alignment and 3D face reconstruction with application to face recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020. 42 (3): 664- 678. |
| 9 | JIANG D L , HU Y X , YAN S C , et al. Efficient 3D reconstruction for face recognition[J]. Pattern Recognition, 2005. 38 (6): 787- 798. |
| 10 | TANG H L , YIN B C , SUN Y F , et al. Pose-invariant face recognition based on a single view[J]. Journal of Information and Computational Science, 2010. 7 (12): 2369- 2376. |
| 11 | PRABHU U , HEO J , SAVVIDES M . Unconstrained pose-invariant face recognition using 3D generic elastic models[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011. 33 (10): 1952- 1961. |
| 12 | ZHU X Y, LEI Z, YAN J J, et al. High-fidelity pose and expression normalization for face recognition in the wild[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Boston, USA: IEEE, 2015: 787-796. |
| 13 | HASSNER T, HAREL S, PAZ E, et al. Effective face frontalization in unconstrained images[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Boston, USA: IEEE, 2015: 4295-4304. |
| 14 | YIM J, JUNG H, YOO B, et al. Rotating your face using multi-task deep neural network[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Boston, USA: IEEE, 2015: 676-684. |
| 15 | KAN M N, SHAN S G, CHANG H, et al. Stacked progressive auto-encoders (SPAE) for face recognition across poses[C]//2014 IEEE Conference on Computer Vision and Pattern Recognition. Columbus, USA: IEEE, 2014: 1883-1890. |
| 16 | TANG C H, HSU G J, YAP M H. Face recognition with disentangled facial representation learning and data augmentation[C]//2019 IEEE International Conference on Image Processing. Taipei, China: IEEE, 2019: 1670-1674. |
| 18 | PAPATHEODOROU T, RUECKERT D. Evaluation of automatic 4D face recognition using surface and texture registration[C]//6th IEEE International Conference on Automatic Face and Gesture Recognition. Seoul, South Korea: IEEE, 2004: 321-326. |
| 19 | CHANG K J , BOWYER K W , FLYNN P J . Effects on facial expression in 3D face recognition[J]. Proceedings of Spie the International Society for Optical Engineering, 2005. 5779, 132- 143. |
| 20 | FALTEMIER T C , BOWYER K W , FLYNN P J . A region ensemble for 3-D face recognition[J]. IEEE Transactions on Information Forensics & Security, 2008. 3 (1): 62- 73. |
| 21 | MOHAMMADZADE H , HATZINAKOS D . Iterative closest normal point for 3D face recognition[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2013. 35 (2): 381- 397. |
| 22 | VIVEK E P , SUDHA N . Robust Hausdorff distance measure for face recognition[J]. Pattern Recognition, 2007. 40 (2): 431- 442. |
| 23 | YU Y , DA F P , GUO Y F . Sparse ICP with resampling and denoising for 3D face verification[J]. IEEE Transactions on Information Forensics and Security, 2019. 14 (7): 1917- 1927. |
| 24 | QUEIROLO C C , SILVA L , BELLON O R P , et al. 3D face recognition using simulated annealing and the surface interpenetration measure[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2010. 32 (2): 206- 219. |
| 25 | AMBERG B, KNOTHE R, VETTER T. Expression invariant 3D face recognition with a morphable model[C]//20088th IEEE International Conference on Automatic Face & Gesture Recognition. Amsterdam, Netherlands: IEEE, 2008: 1-6. |
| 26 | HAAR T , VELTKAMP R C . Expression modeling for expression-invariant face recognition[J]. Computers & Graphics, 2010. 34 (3): 231- 241. |
| 27 | BOOTH J , ROUSSOS A , PONNIAH A , et al. Large scale 3D morphable models[J]. International Journal of Computer Vision, 2018. 2018 (126): 233- 254. |
| 28 | PAN G , WU Z H . 3D face recognition from range data[J]. International Journal of Image and Graphics, 2005. 5 (3): 573- 593. |
| 29 | EFRATY B , BILGAZYEV E , SHAH S , et al. Profile-based 3D-aided face recognition[J]. Pattern Recognition, 2012. 45 (1): 43- 53. |
| 30 | LI Y , WANG Y , LIU J , et al. Expression-insensitive 3D face recognition by the fusion of multiple subject-specific curves[J]. Neurocomputing, 2018. 275, 1295- 1307. |
| 31 | SAMIR C , SRIVASTAVA A , DAOUDI M . Three-dimensional face recognition using shapes of facial curves[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2006. 28 (11): 1858- 1863. |
| 32 | SAMIR C , SRIVASTAVA A , DAOUDI M , et al. An intrinsic framework for analysis of facial surfaces[J]. International Journal of Computer Vision, 2009. 82 (1): 80- 95. |
| 33 | BERRETTI S , BIMBO A D , PALA P . 3D face recognition using isogeodesic stripes[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2010. 32 (12): 2162- 2177. |
| 34 | DRIRA H , AMOR B B , SRIVASTAVA A , et al. 3D face recognition under expressions, occlusions, and pose variations[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2013. 35 (9): 2270- 2283. |
| 35 | LEI Y J , BENNAMOUN M , HAYAT M , et al. An efficient 3D face recognition approach using local geometrical signatures[J]. Pattern Recognition, 2014. 47 (2): 509- 524. |
| 36 | CHANG K T , BOWYER K W , FLYNN P J . An evaluation of multimodal 2D+3D face biometrics[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2005. 27 (4): 619- 624. |
| 37 | PASSALIS G , PERAKIS P , THEOHARIS T , et al. Using facial symmetry to handle pose variations in real-world 3D face recognition[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2011. 33 (10): 1938- 1951. |
| 38 | LIU P J , WANG Y H , HUANG D , et al. Learning the spherical harmonic features for 3-D face recognition[J]. IEEE Transactions on Image Processing, 2013. 22 (3): 914- 925. |
| 39 | GUPTA S, AGGARWAL K, MARKEY K, et al. 3D face recognition founded on the structural diversity of human faces[C]//2007 IEEE Conference on Computer Vision & Pattern Recognition. Minneapolis, USA: IEEE, 2007: 1-7. |
| 40 | PERAKIS P , PASSALIS G , THEOHARIS T , et al. 3D facial landmark detection under large yaw and expression variations[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2013. 35 (7): 1552- 1564. |
| 41 | EMAMBAKHSH M , EVANS A . Nasal patches and curves for expression-robust 3D face recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017. 39 (5): 995- 1007. |
| 42 | BERRETTI S, DEL A, PALA P. 3D partial face matching using local shape descriptors[C]//Joint ACM Workshop on Human Gesture and Behavior Understanding. New York, USA: ACM, 2011: 65-71. |
| 43 | INAN T , HALICI U . 3-D face recognition with local shape descriptors[J]. IEEE Transactions on Information Forensics & Security, 2012. 7 (2): 577- 587. |
| 44 | SMEETS D , KEUSTERMANS J , VANDERMEULEN D , et al. meshSIFT:Local surface features for 3D face recognition under expression variations and partial data[J]. Computer Vision and Image Understanding, 2013. 117 (2): 158- 169. |
| 45 | LI H B , HUANG D , MORVAN J M , et al. Towards 3D face recognition in the real:A registration-free approach using fine-grained matching of 3D keypoint descriptors[J]. International Journal of Computer Vision, 2015. 113 (2): 128- 142. |
| 46 | ELAIWAT S , BENNAMOUN M , BOUSSAID F , et al. A curvelet-based approach for textured 3D face recognition[J]. Pattern Recognition, 2015. 48 (4): 1235- 1246. |
| 47 | MIAN S , OWENS M B R . Keypoint detection and local feature matching for textured 3D face recognition[J]. International Journal of Computer Vision, 2008. 79 (1): 1- 12. |
| 48 | LEI Y J , GUO Y L , HAYAT M , et al. A two-phase weighted collaborative representation for 3D partial face recognition with single sample[J]. Pattern Recognition, 2016. 52, 218- 237. |
| 50 | HUANG D , ARDABILIAN M , WANG Y , et al. 3-D face recognition using eLBP-based facial description and local feature hybrid matching[J]. IEEE Transactions on Information Forensics & Security, 2012. 7 (5): 1551- 1565. |
| 51 | TANG H L , YIN B C , SUN Y F , et al. 3D face recognition using local binary patterns[J]. Signal Processing, 2013. 93 (8): 2190- 2198. |
| 52 | WERGHI N , BERRETTI S , DEL BIMBOQ A . The mesh-LBP:A framework for extracting local binary patterns from discrete manifolds[J]. IEEE Transactions on Image Processing, 2015. 24 (1): 220- 235. |
| 53 | SOLTANPOUR S , WU Q M J . Weighted extreme sparse classifier and local derivative pattern for 3D face recognition[J]. IEEE Transactions on Image Processing, 2019. 28 (6): 3020- 3033. |
| 54 | WANG Y M , LIU J Z , TANG X O . Robust 3D face recognition by local shape difference boosting[J]. IEEE Transations on Pattern Analysis and Machine Intelligence, 2010. 32 (10): 1858- 1870. |
| 55 | HARIRI W , TABIA H , FARAH N , et al. 3D face recognition using covariance based descriptors[J]. Pattern Recognition Letters, 2016. 78, 1- 7. |
| 56 | KIM D, HERNANDEZ M, CHOI J, et al. Deep 3D face identification[C]//2017 IEEE International Joint Conference on Biometrics (IJCB). Denver, USA: IEEE, 2017: 133-142. |
| 57 | LI H B, SUN J, CHEN L M. Location-sensitive sparse representation of deep normal patterns for expression-robust 3D face recognition[C]//2017 IEEE International Joint Conference on Biometrics (IJCB). Denver, USA: IEEE, 2017: 234-242. |
| 58 | PARKHI O M, VEDALDI A, ZISSERMAN A. Deep face recognition[C]//Proceedings of the British Machine Vision Conference. Swansea, UK: BMVA Press, 2015: 41.1-41.12. |
| 59 | GILANI S Z, MIAN A. Learning from millions of 3D scans for large-scale 3D face recognition[C]//IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake city, USA: IEEE, 2018: 1896-1905. |
| 60 | CAI Y , LEI Y J , YANG M L , et al. A fast and robust 3D face recognition approach based on deeply learned face representation[J]. Neurocomputing, 2019. 363 (21): 375- 397. |
| 61 | HSU G S , LIU Y L , PENG H C , et al. RGB-D-based face reconstruction and recognition[J]. IEEE Transactions on Information Forensics and Security, 2014. 9 (12): 2110- 2118. |
| 62 | GOSWAMI G , VATSA M , SINGH R . RGB-D face recognition with texture and attribute features[J]. IEEE Transations on Information Forensics and Security, 2014. 9 (10): 1629- 1640. |
| 63 | MIN R , KOSE N , DUGELAY J . KinectFaceDB:A Kinect database for face recognition[J]. IEEE Transactions on SMC:Systems, 2014. 44 (11): 1534- 1548. |
| 64 | XU X X , LI W , XU D . Distance metric learning using privileged information for face verification and person reidentification[J]. IEEE Transactions on NNLS, 2015. 26 (12): 3150- 3162. |
| 65 | LI B Y L , XUE M L , MIAN A S , et al. Robust RGB-D face recognition using kinect sensor[J]. Neurocomputing, 2016. 214, 93- 108. |
| 66 | LEE Y, CHEN J C, TSENG C, et al. Accurate and robust face recognition from RGB-D images with a deep learning approach[C]//Proceedings of the British Machine Vision Conference. York, UK.: IEEE, 2016: 1-14. |
| 67 | ZHANG H, HAN H, CUI J Y, et al. RGB-D face recognition via deep complementary and common feature learning[C]//IEEE International Conference on Automatic Face & Gesture Recognition. Xi'an, China: IEEE, 2018: 8-15. |
| 68 | JIANG L , ZHANG J Y , DENG B L . Robust RGB-D face recognition using attribute-aware loss[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019. |
| 69 | LUO C W , ZHANG J Y , YU J , et al. Real-time head pose estimation and face modeling from a depth image[J]. IEEE Transactions on Multimedia, 2019. 21 (10): 2473- 2481. |
| 70 | MEYER G P, DO M N. Real-time 3D face verification with a consumer depth camera[C]//201815th Conference on Computer &Robot Vision. Toronto, Canada: IEEE, 2018: 71-79. |
| 71 | KIM D, CHOI J, LEKSUT J T, et al. Accurate 3D face modeling and recognition from RGB-D stream in the presence of large pose changes[C]//2016 IEEE International Conference on Image Processing. Phoenix, USA: IEEE, 2016: 3011-3015. |
| 72 | MU G D, HUANG D, HU G S, et al. Led3D: A lightweight and efficient deep approach to recognizing low-quality 3D faces[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach, USA: IEEE, 2019: 5766-5775. |
| 73 | LI H B , SUN J , XU Z B , et al. Multimodal 2D+3D facial expression recognition with deep fusion convolutional neural network[J]. IEEE Transactions on Multimedia, 2017. 19 (12): 2816- 2831. |
| 74 | SU H, MAJI S, KALOGERAKIS E, et al. Multi-view convolutional neural networks for 3D shape recognition[C]//2015 IEEE International Conference on Computer Vision. Santiago, Chile: IEEE, 2015: 945-953. |
| 75 | SHI B G , BAI S , ZHOU Z C , et al. DeepPano:A deep panoramic representation for 3-D shape recognition[J]. IEEE Signal Processing Letters, 2015. 22 (12): 2339- 2343. |
| 76 | WU Z R, SONG S R, KHOSLA A, et al. 3D ShapeNets: A deep representation for volumetric shapes[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston, USA: IEEE, 2015: 1912-1920. |
| 77 | QI C R, SU H, NIESSNER M, et al. Volumetric and multi-view CNNs for object classification on 3D data[C]//IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA: IEEE, 2016: 5648-5656. |
| 78 | HAN Z Z , LIU Z B , HAN J W , et al. Unsupervised 3D local feature learning by circle convolutional restricted boltzmann machine[J]. IEEE Transactions on Image Processing, 2016. 25 (11): 5331- 5344. |
| 79 | QI C, SU H, MO K C, et al. PointNet: Deep learning on point sets for 3D classification and segmentation[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, USA: IEEE, 2017: 77-85. |
| 80 | BENABDELKADER C , GRIFFIN P A . Comparing and combining depth and texture cues for face recognition[J]. Image & Vision Computing, 2005. 23 (3): 339- 352. |
| 81 | KUSUMA G P , CHUA C S . PCA-based image recombination for multimodal 2D+3D face recognition[J]. Image and Vision Computing, 2011. 29 (5): 306- 316. |
| 82 | XU C H , LI S , TAN T N , et al. Automatic 3D face recognition from depth and intensity Gabor features[J]. Pattern Recognition, 2009. 42 (9): 1895- 1905. |
| 83 | CUI J Y, HAN H, SHAN S G, et al. RGB-D face recognition: A comparative study of representative fusion schemes[C]//Chinese Conference on Biometric Recognition. Urumqi, China: IEEE, 2018: 358-366. |
| 84 | PHILLIPS P J, FLYNN P J, SCRUGGS T, et al. Overview of the face recognition grand challenge[C]//2005 IEEE Conference on Computer Vision and Pattern Recognition. San Diego, USA: IEEE, 2005: 947-954. |
| 85 | XU C H, TAN T N, LI S, et al. Learning effective intrinsic features to boost 3D-based face recognition[C]//ECCV 2006, Lecture Notes in Computer Science. Graz, Austria: Springer-Verlag, 2006: 416-427. |
| 86 | YIN L J, WEI X Z, SUN Y, et al. A 3D facial expression database for facial behavior research[C]//7th International Conference on Automatic Face and Gesture Recognition. Southampton, UK: IEEE, 2006: 211-216. |
| 87 | SAVRAN A, NESE A, HAMDI D, et al. Bosphorus database for 3D face analysis[C]//Biometrics and Identity Management, First European Workshop. Roskilde, Denmark: IEEE, 2008: 47-56. |
| 88 | COLOMBO A, CUSANO C, SCHETTINI R. UMB-DB: A database of partially occluded 3D faces[C]//2011 IEEE International Conference on Computer Vision Workshops. Barcelona, Spain: IEEE, 2011: 2113-2119. |
| 89 | ZHANG J J, HUANG D, WANG Y H, et al. Lock3DFace: A large-scale database of low-cost Kinect 3D faces[C]//International Conference on Biometrics (ICB). Halmstad, Sweden: IEEE, 2016: 1-8. |
/
| 〈 |
|
〉 |
