Abstract：In recent years, the rapid development of generative adversarial networks (GAN) has made synthesized images more and more realistic, which poses great threats to individuals and society. Existing research has focused on passively identifying deepfakes, but real-world applications are usually insufficiently general and robust. This paper presents a method for deepfake provenance and forensics. Deepfakes hide secret information in facial images to track the source of the forged image. An end-to-end deep neural network was designed to include an embedding network, a GAN simulator, and a recovery network. The embedding network embeds the secret information in the picture while the recovery network extracts the information. The GAN simulator simulates various GAN-based image transformations. The average normalized cross correlation coefficient (NCC) of the restored images after tampering with known GANs is higher than 0.9 and the average NCC reaches around 0.8 with tampering by unknown GANs, which shows good robustness and generalization. In addition, the secret embedded information is well concealed and the average peak signal to noise ratio (PSNR) is about 30 dB.
[1] GOODFELLOW I, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial networks[J]. Communications of the ACM, 2020, 63(11):139-144. [2] VERDOLIVA L. Media forensics and DeepFakes:An overview[J]. IEEE Journal of Selected Topics in Signal Processing, 2020, 14(5):910-932. [3] LI Y Z, YANG X, SUN P, et al. Celeb-DF:A large-scale challenging dataset for deepfake forensics[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle, WA, USA:IEEE Press, 2020:3207-3216. [4] QIAN Y Y, YIN G J, SHENG L, et al. Thinking in frequency:Face forgery detection by mining frequency-aware clues[M]//VEDALDI A, BISCHOF H, BROX T, et al. 16th European Conference Proceedings of the European Conference on Computer Vision. Cham, Switzerland:Springer, 2020:86-103. [5] CARLINI N, FARID H. Evading deepfake-image detectors with white and black-box attacks[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Seattle, WA, USA:IEEE Press, 2020:658-659. [6] HE Z L, ZUO W M, KAN M N, et al. AttGAN:Facial attribute editing by only changing what you want[J]. IEEE Transactions on Image Processing, 2019, 28(11):5464-5478. [7] LIU M, DING Y K, XIA M, et al. STGAN:A unified selective transfer network for arbitrary image attribute editing[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach, CA, USA:IEEE Press, 2019:3673-3682. [8] CHOI Y, CHOI M, KIM M, et al. StarGAN:Unified generative adversarial networks for multi-domain image-to-image translation[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, UT, USA:IEEE Press, 2018:8789-8797. [9] LIU Z W, LUO P, WANG X G, et al. Deep learning face attributes in the wild[C]//2015 IEEE International Conference on Computer Vision. Santiago, Chile:IEEE Press, 2015:3730-3738. [10] YANG X, LI Y Z, LYU S W. Exposing deep fakes using inconsistent head poses[C]//ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing. Brighton, UK:IEEE Press, 2019:8261-8265. [11] ZHANG X, KARAMAN S, CHANG S F. Detecting and simulating artifacts in GAN fake images[C]//2019 IEEE International Workshop on Information Forensics and Security. Delft, Netherlands:IEEE Press, 2019:1-6. [12] DURALL R, KEUPER M, PFREUNDT F J, et al. Unmasking deepfakes with simple features[J/OL].[2021-10-18]. https://arxiv.org/pdf/1911.00686.pdf. [13] 郝彦军, 朱琴, 王丽娜, 等. 数字水印演化设计[J]. 计算机工程, 2006, 32(6):157-159. HAO Y J, ZHU Q, WANG L N, et al. Digital watermarking design with evolutionary computation[J]. Computer Engineering, 2006, 32(6):157-159. (in Chinese) [14] MUYCO S D, HERNANDEZ A A. Least significant bit hash algorithm for digital image watermarking authentication[C]//Proceedings of the 2009 5th International Conference on Computing and Artificial Intelligence. New York, NY, USA:ACM, 2019:150-154. [15] XU H C, KANG X B, WANG Y H, et al. Exploring robust and blind watermarking approach of colour images in DWT-DCT-SVD domain for copyright protection[J]. International Journal of Electronic Security and Digital Forensics, 2018, 10(1):79-96. [16] ZHANG C N, BENZ P, KARJAUV A, et al. UDH:Universal deep hiding for steganography, watermarking, and light field messaging[J]. Advances in Neural Information Processing Systems, 2020, 33:10223-10234.
2022, Vol. 62 
