Review-aware heterogeneous variational autoencoder recommendation model
LIU Shudong1,2, ZHANG Jiani1,2, CHEN Xu1,2
1. Centre for Artificial Intelligence and Applied Research, Zhongnan University of Economics and Law, Wuhan 430073, China; 2. School of Information and Security Engineering, Zhongnan University of Economics and Law, Wuhan 430073, China
Abstract：With the advancement of research and development in recommendation systems, more attention has been paid to the precise recommendation of personalized information. The traditional method of collaborative filtering cannot meet the demand due to the scarcity of data in recommendation; thus, contextual information has been introduced to recommendation systems. Review text information containing user preferences is also widely used to alleviate data sparseness and cold start problems. As an unsupervised learning method, the autoencoder performs well in anomaly detection, face recognition, data augmentation, and data generation. The variational autoencoder can learn the distribution of latent vectors of users and items via neural networks. At present, only a few researchers are working for review-aware recommendations using variational autoencoder. This paper proposes a review-aware heterogeneous variational autoencoder recommendation model that introduces comment context information into the variational autoencoder through attention mechanism and neural network. The learning about latent feature distribution of the rating information by the variational autoencoder is retained, and feature fusion is performed in the early and late stages to construct a multimodal heterogeneous variational autoencoder model. Besides, the compound prior term and the balance factor calculation term for multimodal model training are further optimized. The experimental results showed that the proposed model outperforms the state-of-the-art other baseline models in recall and the normalized cumulative gain.
[1] ADOMAVICIUS G, TUZHILIN A. Personalization technologies:A process-oriented perspective[J]. Communications of the ACM, 2005, 48(10):83-90. [2] ADOMAVICIUS G, TUZHILIN A. Toward the next generation of recommender systems:A survey of the state-of-the-art and possible extensions[J]. IEEE Transactions on Knowledge and Data Engineering, 2005, 17(6):734- 749. [3] CHEN H Y, LI J. Adversarial tensor factorization for context-aware recommendation[C]//Proceedings of the 13th ACM Conference on Recommender Systems. New York, USA:Association for Computing Machinery, 2019:363-367. [4] XIN X, CHEN B, HE X N, et al. CFM:Convolutional factorization machines for context-aware recommendation[C]//Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence. Macao, China:AAAI, 2019:3926-3932. [5] WU L, SUN P J, FU Y J, et al. A neural influence diffusion model for social recommendation[C]//Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval. New York, USA:ACM, 2019:235-244. [6] FAN W Q, MA Y, LI Q, HE Y, ZHAO E. Graph neural networks for social recommendation[C]//Proceedings of the World Wide Web Conference. New York, USA:Association for Computing Machinery, 2019:417-426. [7] 王立才, 孟祥武, 张玉洁. 上下文感知推荐系统[J]. 软件学报, 2012, 23(1):1-20. WANG L C, MENG X W, ZHANG Y J. Context-aware recommender systems[J]. Journal of Software, 2012, 23(1):1-20. (in Chinese) [8] KULKARNI S, RODD S F. Context aware recommendation systems:A review of the state of the art techniques[J]. Computer Science Review, 2020, 37:100255. [9] 孟祥武, 刘树栋, 张玉洁, 等. 社会化推荐系统研究[J]. 软件学报, 2015, 26(6):1356-1372. MENG X W, LIU S D, ZHANG Y J, et al. Research on social recommender systems[J]. Journal of Software, 2015, 26(6):1356-1372. (in Chinese) [10] RENDLE S. Factorization machines[C]//Proceedings of the 10th IEEE International Conference on Data Mining. New York, USA:IEEE Press, 2010:995-1000. [11] SHEN R P, ZHANG H P, YU H, et al. Sentiment based matrix factorization with reliability for recommendation[J]. Expert Systems with Applications, 2019, 135:249-258. [12] LIAN D F, ZHAO C, XIE X, et al. GeoMF:Joint geographical modeling and matrix factorization for point-of-interest recommendation[C]//Proceedings of the 20th ACM SIGKDD Interna-tional Conference on Knowledge Discovery and Data Mining. New York, USA:Association for Computing Machinery, 2014:831-840. [13] GUO H F, TANG R M, YE Y M, et al. DeepFM:A factorization-machine based neural network for CTR prediction[C]//Proceedings of the 26th International Joint Conference on Artificial Intelligence (IJCAI). California, USA:IJCAI.org, 2017:1725-1730. [14] HE X N, LIAO L Z, ZHANG H W, et al. Neural collaborative filtering[C]//Proceedings of the 26th International Conference on World Wide. Republic and Canton of Geneva, Switzerland:International World Wide Web Conferences Steering Committee, 2017:173-182. [15] 袁非牛, 章琳, 史劲亭, 等. 自编码神经网络理论及应用综述[J]. 计算机学报, 2019, 42(1):203-230. YUAN F N, ZHANG L, SHI J T, et al. Theories and applications of auto-encoder neural networks:A literature survey[J]. Chinese Journal of Computers, 2019, 42(1):203-230. (in Chinese) [16] LI H, WANG Y L, LYU Z Y, et al. Multi-task learning for recommendation over heterogeneous information network[J/OL] IEEE Transactions on Knowledge and Data Engineering, (2020-3-31). https://ieeexplore. ieee.org/document/9051843. [17] ZHANG G J, LIU Y, JIN X N. A survey of auto-encoder-based recommender systems[J]. Frontiers of Computer Science, 2020, 14(2):430-450. [18] WANG H, SHI X J, YEUNG D Y. Collaborative recurrent auto-encoder:recommend while learning to fill in the blanks[C]//Proceedings of the 30th International Conference on Neural Information Processing Systems. Red Hook, NY, USA:Curran Associates Inc., 2016:415-423. [19] LI X P, SHE J. Collaborative variational auto-encoder for recommender systems[C]//Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York, USA:Association for Computing Machinery, 2017:305-314. [20] ZHUANG F Z, ZHANG Z Q, QIAN M, et al. Representation learning via dual-autoencoder for recommendation[J]. Neural Networks, 2017, 90:83-89. [21] ZHU Z, WANG J, CAVERLEE J. Improving top-k recommendation via joint collaborative auto-encoders[C]//Proceedings of The World Wide Web Conference. New York, USA:Association for Computing Machinery, 2019:3483-3492. [22] LACIC E, REITER-HAAS M, KOWALD D, et al.. Using auto-encoders for session-based job recommendations[J]. User Modeling and User-Adapted Interaction, 2020, 30(4):617-658. [23] JHAMB Y, EBESU T, FANG Y. Attentive contextual denoising auto-encoder for recommendation[C]//Proceedings of the 2018 ACM SIGIR International Conference on Theory of Information Retrieval. New York, USA:ACM, 2018:27-34. [24] ZHOU J P, CHENG Z Y, PEREZ F, et al. TAFA:Two-headed attention fused auto-encoder for context-aware recommendations[C]//Proceedings of Fourteenth ACM Conference on Recommender Systems. New York, USA:Association for Computing Machinery, 2020:338-347. [25] DENG S G, HUANG L T, XU G D, et al. On deep learning for trust-aware recommendations in social networks[J]. IEEE Transactions on Neural Networks and Learning Systems, 2017, 28(5):1164-1177. [26] WANG M Q, WU Z Y, SUN X X, et al. Trust-aware collaborative filtering with a denoising auto-encoder[J]. Neural Processing Letters, 2019, 49(2):835-849. [27] BELLINI V, SCHIAVONE A, DI NOIA T, et al. Knowledge-aware auto-encoders for explainable recommender systems[C]//Proceedings of the 3rd Workshop on Deep Learning for Recommender Systems. New York, USA:Association for Computing Machinery, 2018:24-31. [28] CHEN Y F, DE RIJKE M. A collective variational auto-encoder for top-n recommendation with side information[C]//Proceedings of the 3rd Workshop on Deep Learning for Recommender Systems. New York, USA:Association for Computing Machinery, 2018:3-9. [29] GALKE L, MAI F, VAGLIANO I, et al. Multi-modal adversarial auto-encoders for recommendations of citations and subject labels[C]//Proceedings of the 26th Conference on User Modeling, Adaptation and Personalization. New York, USA:Association for Computing Machinery, 2018:197-205. [30] YANG C, YU X H, LIU Y, et al. Collaborative filtering with weighted opinion aspects[J]. Neurocomputing, 2016, 210:185-196. [31] WANG Y H, LIU Y, YU X H. Collaborative filtering with aspect-based opinion mining:A tensor factorization approach[C]//Proceedings of the 2012 IEEE 12th International Conference on Data Mining. Brussels, Belgium:IEEE Press, 2012:1152-1157. [32] NIE Y P, LIU Y, YU X H. Weighted aspect-based collaborative filtering[C]//Proceedings of the 37th International ACM SIGIR Conference on Research & Development in Information Retrieval (SIGIR). New York, USA:ACM, 2014:1071-1074. [33] CHENG Z Y, DING Y, ZHU L, KANKANHALLI M. Aspect-aware latent factor model:Rating prediction with ratings and reviews[C]//Proceedings of the 2018 World Wide Web Conference. New York, USA:Association for Computing Machinery, 2018:639-648. [34] WANG W, WANG H W. Opinion-enhanced collaborative filtering for recommender systems through sentiment analysis[J]. New Review of Hypermedia and Multimedia, 2015, 21(3-4):278-300. [35] ALMAHAIRI A, KASTNER K, CHO K, et al. Learning distributed representations from reviews for collaborative filtering[C]//Proceedings of the 9th ACM Conference on Recommender Systems. New York, USA:Association for Computing Machinery, 2015:147-154. [36] HYUN D, PARK C, YANG M C. Review sentiment-guided scalable deep recommender system[C]//Proceedings of the 41st International ACM SIGIR Conference on Research & Development in Information Retrieval. New York, USA:ACM, 2018:965-968. [37] TAY Y, LUU A T, HUI S C. Multi-pointer co-attention networks for recommendation[C]//Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. New York, USA:Association for Computing Machinery, 2018:2309-2318. [38] GUAN X Y, CHENG Z Y, HE X N, et al. Attentive aspect modeling for review-aware recommendation[J]. ACM Transactions on Information Systems, 2019, 37(3):1-27. [39] HUANG G, LIU Z, VAN DER MAATEN L, et al. Densely connected convolutional networks[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. New York, USA:IEEE Press, 2017:4700-4708. [40] SHENBIN I, ALEKSEEV A, TUTUBALINA E, et al. RecVAE:A new variational autoencoder for top-n recommendations with implicit feedback[C]//Proceedings of the 13th International Conference on Web Search and Data Mining. New York, USA:Association for Computing Machinery, 2020:528-536. [41] JÄRVELIN K, KEKÄLÄINEN J. Cumulated gain-based evaluation of IR techniques[J]. ACM Transactions on Information Systems, 2002, 20(4):422-446. [42] KIM D, PARK C, OH J, et al. Convolutional matrix factorization for document context-aware recommendation[C]//Proceedings of the 10th ACM Conference on Recommender Systems. New York, USA:Association for Computing Machinery, 2016:233-240. [43] MA C, KANG P, WU B, et al. Gated attentive-autoencoder for content-aware recommendation[C]//Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining. New York, USA:Association for Computing Machinery, 2019:519-527.
2022, Vol. 62 
