A Chinese keyphrase extraction method for multimodal information enhancement representation

ZHOU Xuanyu; LIU Lin; LU Xiao; LI Xuan; ZHANG Siming

doi:10.16511/j.cnki.qhdxxb.2024.27.015

Journal of Tsinghua University(Science and Technology) >

2024 , Vol. 64 >Issue 10: 1785 - 1796

DOI: https://doi.org/10.16511/j.cnki.qhdxxb.2024.27.015

SPECIAL SECTION: BIG DATA ANALYTICS

A Chinese keyphrase extraction method for multimodal information enhancement representation

ZHOU Xuanyu ,
LIU Lin ,
LU Xiao ,
LI Xuan ,
ZHANG Siming

Expand

1. Key Laboratory of Big Data Research and Application for Basic Education, Hunan Normal University, Changsha 410006, China;
2. College of Engineering and Design, Hunan Normal University, Changsha 410006, China

Received date: 2024-01-09

Online published: 2024-09-20

Fold

Abstract

[Objective] At present, China is undergoing a critical digital transformation in education. This shift has led to an explosive growth of educational content online, presenting a challenge for researchers who find it increasingly difficult to sift through massive amounts of text data. The necessity to quickly grasp important information has made keyphrase extraction an invaluable tool. Keyphrase extraction automates the process of identifying words or phrases that encapsulate the main themes of a text, proving critical for text retrieval, text summary, and other tasks. Despite its importance, the current keyphrase extraction tasks mainly rely on pretrained language models to obtain text representation. These models are often trained based on a generic text corpus and struggle to adapt to specific domains according to the characteristics of downstream tasks owing to their limited ability to capture the subtle semantic representation of single-mode information. Therefore, developing methods for accurate and efficient keyphrase extraction from massive texts remains a pressing research challenge. [Methods] This paper presents a novel approach for Chinese keyphrase extraction, dubbed multimodal information enhancement representation for keyphrase extraction (MIEnhance-KPE). Our method first deconstructs characters into radicals using a character splitting dictionary and extracts radical features through a convolutional neural network. At the same time, we integrate a trainable adapter layer between the transformer layers of a pretrained language model. Through the above operations, the bottom level semantic features of the pretrained language model and radical features are fully integrated to obtain a domain adaptive text representation. Characters are then transformed into glyph images representing different periods in history and writing styles. Subsequently, we employ group convolution to extract the glyphic features of these characters. Meanwhile, a cross-attention mechanism is used to fuse the glyphic and text features, yielding richer and more comprehensive semantic representations. The final step involves using a conditional random field model to learn the relationship between the fused features and labels. Through sequence labeling, we identify candidate keyphrases, ranking them based on position and word frequency weight to determine the most relevant keyphrases. [Results] MIEnhance-KPE's performance was tested using two datasets: the published Chinese Scientific Literature (CSL) and the self-constructed Chinese Education Keyphrase Extraction Dataset (CEKED). Our method demonstrated a substantial improvement compared to the most advanced keyphrase extraction methods, with F values increasing by 15.71% and 3.40% on the CSL and CEKED datasets, respectively. Ablation experiments further confirmed the effectiveness of both the domain adaptive module and the visual semantic enhancement module in enhancing keyphrase extraction accuracy. In addition, this paper explored various methods for fusing glyphic and semantic features, concluding that the cross-attention mechanism excels in adaptively merging different features to improve task accuracy. [Conclusions] The MIEnhance-KPE proposed in this paper can considerably improve the accuracy of keyphrase extraction tasks. This aids educational researchers in quickly locating relevant literature and understanding the cutting-edge trends of educational development. Additionally, MIEnhance-KPE introduces a novel approach to literature analysis in the educational sector. It provides a solid data foundation for examining the motivation of educational reform and innovation, thereby accelerating the digital transformation process in education.

Key words： Chinese keyphrase extraction; multimodal information; multigranularity semantic features; cross-attention mechanism; domain adaptation

Cite this article

ZHOU Xuanyu , LIU Lin , LU Xiao , LI Xuan , ZHANG Siming . A Chinese keyphrase extraction method for multimodal information enhancement representation[J]. Journal of Tsinghua University(Science and Technology), 2024 , 64(10) : 1785 -1796 . DOI: 10.16511/j.cnki.qhdxxb.2024.27.015

References

[1] 赵京胜, 朱巧明, 周国栋, 等. 自动关键词抽取研究综述[J]. 软件学报, 2017, 28(9): 2431-2449. ZHAO J S, ZHU Q M, ZHOU G D, et al. Review of research in automatic keyword extraction [J]. Journal of Software, 2017, 28(9): 2431-2449. (in Chinese)
[2] 胡少虎, 张颖怡, 章成志. 关键词提取研究综述[J]. 数据分析与知识发现, 2021, 5(3): 45-59. HU S H, ZHANG Y Y, ZHANG C Z. Review of keyword extraction studies [J]. Data Analysis and Knowledge Discovery, 2021, 5(3): 45-59. (in Chinese)
[3] DING N, QIN Y J, YANG G, et al. Parameter-efficient fine-tuning of large-scale pre-trained language models [J]. Nature Machine Intelligence, 2023, 5(3): 220-235.
[4] YIN M W, MOU C J, XIONG K N, et al. Chinese clinical named entity recognition with radical-level feature and self-attention mechanism [J]. Journal of Biomedical Informatics, 2019, 98: 103289.
[5] WANG S N, KHABSA M, MA H. To pretrain or not to pretrain: examining the benefits of pretrainng on resource rich tasks [C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. New York, USA: ACL Press, 2020: 2209-2213.
[6] MENG Y X, WU W, WANG F, et al. Glyce: Glyph-vectors for Chinese character representations [C]//Proceedings of the 33rd International Conference on Neural Information Processing Systems (NeurIPS 2019). Vancouver, Canada: Curran Associates Inc., 2019: 2746-2757.
[7] XUAN Z Y, BAO R, JIANG S Y. FGN: Fusion glyph network for Chinese named entity recognition [C]//5th China Conference on Knowledge Graph and Semantic Computing: Knowledge Graph and Cognitive Intelligence. Nanchang, China: Springer Press, 2021: 28-40.
[8] SALTON G, BUCKLEY C. Term-weighting approaches in automatic text retrieval [J]. Information Processing & Management, 1988, 24(5): 513-523.
[9] 庞庆华, 董显蔚, 周斌, 等. 基于情感分析与TextRank的负面在线评论关键词抽取[J]. 情报科学, 2022, 40(5): 111-117. PANG Q H, DONG X W, ZHOU B, et al. Keyword extraction of negative online reviews based on sentiment analysis [J]. Information Science, 2022, 40(5): 111-117. (in Chinese)
[10] 马慧芳, 刘芳, 夏琴, 等. 基于加权超图随机游走的文献关键词提取算法[J]. 电子学报, 2018, 46(6): 1410-1414. MA H F, LIU F, XIA Q, et al. Extraction algorithm based on weighted hypergraph random walk [J]. Acta Electronica Sinica, 2018, 46(6): 1410-1414. (in Chinese)
[11] TERRYN A R, DROUIN P, HOSTE V, et al. Analysing the impact of supervised machine learning on automatic term extraction: HAMLET vs TermoStat [C]//Proceedings of the International Conference on Recent Advances in Natural Language Processing. Varna, Bulgaria: INCOMA Ltd. Press, 2019: 1012-1021.
[12] ZHANG C Z, WANG H L, LIU Y, et al. Automatic keyword extraction from documents using conditional random fields [J]. Journal of Computational Information Systems, 2008, 4(3): 1169-1180.
[13] ARDIANSYAH S, MAJID M A, ZAIN J M. Knowledge of extraction from trained neural network by using decision tree [C]//Proceedings of 20162nd International Conference on Science in Information Technology (ICSITech). Balikpapan, Indonesia: IEEE Press, 2016: 220-225.
[14] CHAN H P, CHEN W, WANG L, et al. Neural keyphrase generation via reinforcement learning with adaptive rewards [C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Florence, Italy: ACL Press, 2019: 2163-2174.
[15] CHEN W, CHAN H P, LI P J, et al. Exclusive hierarchical decoding for deep keyphrase generation [C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. New York, USA: ACL Press, 2020: 1095-1105.
[16] ZHANG Y Y, ZHANG C Z. Using human attention to extract keyphrase from microblog post [C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Florence, Italy: ACL Press, 2019: 5867-5872.
[17] 曾道建, 童国维, 戴愿, 等.基于序列到序列模型的法律问题关键词抽取[J].清华大学学报(自然科学版), 2019, 59(04): 256-261. ZENG D J, TONG G W, DAI Y, et al. Keyphrase extraction for legal questions based on a sequence to sequence model [J]. Journal of Tsinghua University (Science and Technology), 2019, 59(4): 256-261. (in Chinese)
[18] ZHANG Q, WANG Y, GONG Y Y, et al. Keyphrase extraction using deep recurrent neural networks on Twitter [C]//Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. Austin, Texas: ACL Press, 2016: 836-845
[19] 孙新, 盖晨, 申长虹, 等. 基于短语向量和主题加权的关键词抽取方法[J]. 电子学报, 2021, 49(9): 1682-1690. SUN X, GAI C, SHEN C H, et al. The theme-weighted keyphrase extraction algorithm based on phrase embedding [J]. Acta Electronica Sinica, 2021, 49(9): 1682-1690. (in Chinese)
[20] SAHRAWAT D, MAHATA D, ZHANG H M, et al. Keyphrase extraction as sequence labeling using contextualized embeddings [C]//42nd European Conference on IR Research on Advances in Information Retrieval (ECIR). Lisbon, Portugal: Springer, 2020: 328-335.
[21] ZHANG L H, CHEN Q, WANG W, et al. MDERank: A masked document embedding rank approach for unsupervised keyphrase extraction [C]//Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. Dublin, Ireland: ACL Press, 2022: 396-409.
[22] SUN S, XIONG C Y, LIU Z H, et al. Capturing global informativeness in open domain keyphrase extraction [C]//Proceedings of the Natural Language Processing and Chinese Computing. QingDao, China: Springer. Press, 2021: 275-287.
[23] SONG M Y, FENG Y, JING L P. Hyperbolic relevance matching for neural keyphrase extraction [C]//Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Seattle, United States: ACL Press, 2022: 396-409.
[24] DONG C H, ZHANG J J, ZONG C Q, et al. Character-based LSTM-CRF with radical-level features for Chinese named entity recognition [C]//Natural Language Under-standing and Intelligent Applications: 5th CCF Conference on Natural Language Processing and Chinese Computing and 24th International Conference on Computer Processing of Oriental Languages. Kunming, China: Springer International Press, 2016: 239-250.
[25] HAN X K, YUE Q, CHU J, et al. Multi-feature fusion transformer for Chinese named entity recognition [C]//202241st Chinese Control Conference. Hefei, China: IEEE Press, 2022: 4227-4232.
[26] WU S, SONG X N, FENG Z H. MECT: Multi-metadata embedding based cross-transformer for Chinese named entity recognition [C]//Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing. New York, USA: ACL Press, 2021: 1529-1539.
[27] CHEN H Y, YU S H, LIN S D. Glyph2Vec: Learning Chinese out-of-vocabulary word embedding from glyphs [C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. New York, USA: ACL Press, 2020: 2865-2871.
[28] CUI Y M, CHE W X, LIU T, et al. Pre-training with whole word masking for Chinese BERT [C]//Proceedings of IEEE/ACM Transactions on Audio, Speech, and Language Processing. Balikpapan, Indonesia: IEEE Press, 2021: 3504-3514.
[29] SUN Y, QIU H, ZHENG Y, et al. SIFRank: a new baseline for unsupervised keyphrase extraction based on pre-trained language model [J]. IEEE Access, 2020, 8: 10896-10906.
[30] LI Y D, ZHANG Y Q, ZHAO Z, et al. CSL: A large-scale Chinese scientific literature dataset [C]//Proceedings of the 29th International Conference on Computational Linguistics. Gyeongju, Republic of Korea: International Committee on Computational Linguistics Press, 2022: 3917-3923.
[31] WU X Y, BOLIVAR A. Keyword extraction for contextual advertisement [C]//Proceedings of the 17th international conference on World Wide Web. New York, USA: ACM Press, 2008: 1195-1196.
[32] WANG J J, XU C S, CHNG E, et al. Sports highlight detection from keyword sequences using HMM [C]//2004 IEEE International Conference on Multimedia and Expo. Taipei, China: IEEE, 2004: 599-602.
[33] GERO Z, HO J. Word centrality constrained representation for keyphrase extraction [C]//Proceedings of the 20th Workshop on Biomedical Language Processing. New York, USA: ACL Press, 2021: 155-161.
[34] SONG M Y, XIAO L, JING L P. Learning to extract from multiple perspectives for neural keyphrase extraction[J]. Computer Speech & Language, 2023, 81: 101502.

Options

Outlines

模态框（Modal）标题

Abstract

Cite this article

References

Visited