| COMPUTER SCIENCE AND TECHNOLOGY |
|
|
|
|
|
Human-machine conversation system for chatting based on scene and topic |
| LU Sicong, LI Chunwen |
| Department of Automation, Tsinghua University, Beijing 100084, China |
|
|
|
|
Abstract Human-machine conversation plays an important role in natural language processing and artificial intelligence. Human-machine conversation can be divided into the question answering system, task-oriented conversation, and chatting system according to the purpose of use. Among them, the chatting system usually requires higher personification. Based on the sequence transformation model of the long short-term memory network, the topic network is introduced in this study to explicitly extract the scene and topic information from the conversation, and this higher-level feature, which does not change with the word order, is inputted to the structure of the conversation model to guide the decoding and prediction processes together with the attention mechanism. Because of the difficulty of obtaining the topic information in advance, the topic network is modeled as an unsupervised learning structure. Thus, the training process needs to be divided into three steps. The experimental results show that the model can significantly improve the quality of the chatting system with appropriate training methods and structural parameters.
|
| Keywords
artificial intelligence
natural language processing
human-machine conversation
machine learning
topic network
|
|
Issue Date: 26 April 2022
|
|
|
[1] SHAWAR B A, ATWELL E S. Using corpora in machine-learning chatbot systems[J]. International Journal of Corpus Linguistics, 2005, 10(4):489-516.
[2] 易顺明, 胡振宇. 中文聊天机器人原型系统的设计[J]. 沙洲职业工学院学报, 2007, 10(2):5-9. YI S M, HU Z Y. The prototype design for the Chinese chat robots[J]. Journal of Shazhou Professional Institute of Technology, 2007, 10(2):5-9. (in Chinese)
[3] SUTSKEVER I, VINYALS O, LE Q V. Sequence to sequence learning with neural networks[EB/OL]. (2014-12-14)[2021-05-01]. https://arxiv.org/abs/1409.3215v3.
[4] SUNDERMEYER M, NEY H, SCHLVTER R. From feedforward to recurrent LSTM neural networks for language modeling[J]. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2015, 23(3):517-529.
[5] LUONG M T, PHAM H, MANNING C D. Effective approaches to attention-based neural machine translation[EB/OL]. (2015-09-20)[2021-05-01]. https://arxiv.org/abs/1508.04025v5.
[6] 王红, 史金钏, 张志伟. 基于注意力机制的LSTM的语义关系抽取[J]. 计算机应用研究, 2018, 35(5):1417-1420, 1440. WANG H, SHI J C, ZHANG Z W. Text semantic relation extraction of LSTM based on attention mechanism[J]. Application Research of Computers, 2018, 35(5):1417-1420, 1440. (in Chinese)
[7] NIO L, SAKTI S, NEUBIG G, et al. Conversation dialog corpora from television and movie scripts[C]//2014 17th Oriental Chapter of the International Committee for the Co-ordination and Standardization of Speech Databases and Assessment Techniques (COCOSDA). Phuket, Thailand:IEEE Press, 2014:1-4.
[8] 曾小芹. 基于Python的中文结巴分词技术实现[J]. 信息与电脑(理论版), 2019, 31(18):38-39, 42. ZENG X Q. Technology implementation of Chinese jieba segmentation based on python[J]. China Computer & Communication, 2019, 31(18):38-39, 42. (in Chinese)
[9] RONG X. Word2vec parameter learning explained[EB/OL]. (2016-06-05)[2021-05-01]. https://arxiv.org/abs/1411.2738v4.
[10] 张伟男, 张杨子, 刘挺. 对话系统评价方法综述[J]. 中国科学:信息科学, 2017, 47(8):953-966. ZHANG W N, ZHANG Y Z, LIU T. Survey of evaluation methods for dialogue systems[J]. Scientia Sinica (Informationis), 2017, 47(8):953-966. (in Chinese)
[11] MOLDOVAN D I, TATU M. Natural language question answering system and method utilizing multi-modal logic:US20060053000 A1[P]. 2006-03-09.
[12] 邢超. 智能问答系统的设计与实现[D]. 北京:北京交通大学, 2015. XING C. The design and implementation of intelligent question and answering system[D]. Beijing:Beijing Jiaotong University, 2015. (in Chinese)
[13] WEN T H, VANDYKE D, MRKSIC N, et al. A network-based end-to-end trainable task-oriented dialogue system[EB/OL]. (2017-04-24)[2021-05-01]. https://arxiv.org/abs/1604.04562v3.
[14] 张杰晖. 任务型对话系统的自然语言生成研究[D]. 广州:华南理工大学, 2019. ZHANG J H. Research on natural language generation in task-based dialogue system[D]. Guangzhou:South China University of Technology, 2019. (in Chinese)
[15] MĚKOTA O, GÖKIRMAK M, LAITOCH P. End to end dialogue transformer[EB/OL]. (2020-08-24)[2021-05-01]. https://www.researchgate.net/publication/343849046_End _to_End_Dialogue_Transformer.
[16] THIERGART J, HUBER S, VBELLACKER T. Under-standing emails and drafting responses-An approach using GPT-3[EB/OL]. (2021-02-15)[2021-05-01]. https://arxiv.org/abs/2102.03062v3.
[17] 张献涛, 张猛, 暴筱, 等. 一种提升人机交互对话语料质量与多样性的对话语料库生成方法:CN111026884A[P]. 2020-04-17. ZHANG X T, ZHANG M, BAO X, et al. Dialogue corpus generation method for improving quality and diversity of man-machine interaction dialogue corpora:CN111026884A[P]. 2020-04-17. (in Chinese)
[18] WIKIPEDIA. Long short-term memory[EB/OL]. (2021-03-25)[2021-05-01]. https://en.wikipedia.org/wiki/Long_short-term_memory.
[19] WIKIPEDIA. Attention (machine learning)[EB/OL]. (2021-02-27)[2021-05-01]. https://en.wikipedia.org/wiki/Attention_(machine_learning).
[20] WIKIPEDIA. BLEU[EB/OL]. (2020-11-09)[2021-05-01]. https://en.wikipedia.org/wiki/BLEU.
[21] WIKIPEDIA. Autoencoder[EB/OL]. (2021-03-24)[2021-05-01]. https://en.wikipedia.org/wiki/Autoencoder.
[22] DANESCU-NICULESCU-MIZIL C, LEE L. Chameleons in imagined conversations:A new approach to understanding coordination of linguistic style in dialogs[C]//CMCL 2011:Proceedings of the 2nd Workshop on Cognitive Modeling and Computational Linguistics. Portland, Oregon, USA:Association for Computational Linguistics, 2011:76-87.
[23] GITHUB. Fxsjy/jieba[DB/OL]. (2020-01-20)[2021-05-01]. https://github.com/fxsjy/jieba.
[24] WIKIPEDIA. Beam search[EB/OL]. (2021-03-11)[2021-05-01]. https://en.wikipedia.org/wiki/Beam_search. |
|
Viewed |
|
|
|
Full text
|
|
|
|
|
Abstract
|
|
|
|
|
Cited |
|
|
|
|
| |
Shared |
|
|
|
|
| |
Discussed |
|
|
|
|
2022,
Vol. 62
