人机会话在自然语言处理乃至人工智能领域中起着重要标志性作用，可根据使用目的划分为问答系统、任务型会话、聊天系统等，其中聊天型会话通常具有更高的拟人需求。该文在基于长短期记忆网络的序列变换模型基础上，通过引入话题网络显式抽取会话中的场景与话题信息，并将这种不随语序变化的高层抽象信息扩展到会话模型结构中，与注意力机制共同指导解码预测过程。由于难以事先获取话题信息，话题网络被建模为非监督式学习模型，因此训练过程需分三步进行。实验结果表明，在恰当的训练方法和结构参数下，该模型能够使聊天会话的质量得到明显提升。
Human-machine conversation plays an important role in natural language processing and artificial intelligence. Human-machine conversation can be divided into the question answering system, task-oriented conversation, and chatting system according to the purpose of use. Among them, the chatting system usually requires higher personification. Based on the sequence transformation model of the long short-term memory network, the topic network is introduced in this study to explicitly extract the scene and topic information from the conversation, and this higher-level feature, which does not change with the word order, is inputted to the structure of the conversation model to guide the decoding and prediction processes together with the attention mechanism. Because of the difficulty of obtaining the topic information in advance, the topic network is modeled as an unsupervised learning structure. Thus, the training process needs to be divided into three steps. The experimental results show that the model can significantly improve the quality of the chatting system with appropriate training methods and structural parameters.
[1] SHAWAR B A, ATWELL E S. Using corpora in machine-learning chatbot systems[J]. International Journal of Corpus Linguistics, 2005, 10(4):489-516.
[2] 易顺明, 胡振宇. 中文聊天机器人原型系统的设计[J]. 沙洲职业工学院学报, 2007, 10(2):5-9. YI S M, HU Z Y. The prototype design for the Chinese chat robots[J]. Journal of Shazhou Professional Institute of Technology, 2007, 10(2):5-9. (in Chinese)
[3] SUTSKEVER I, VINYALS O, LE Q V. Sequence to sequence learning with neural networks[EB/OL]. (2014-12-14)[2021-05-01]. https://arxiv.org/abs/1409.3215v3.
[4] SUNDERMEYER M, NEY H, SCHLVTER R. From feedforward to recurrent LSTM neural networks for language modeling[J]. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2015, 23(3):517-529.
[5] LUONG M T, PHAM H, MANNING C D. Effective approaches to attention-based neural machine translation[EB/OL]. (2015-09-20)[2021-05-01]. https://arxiv.org/abs/1508.04025v5.
[6] 王红, 史金钏, 张志伟. 基于注意力机制的LSTM的语义关系抽取[J]. 计算机应用研究, 2018, 35(5):1417-1420, 1440. WANG H, SHI J C, ZHANG Z W. Text semantic relation extraction of LSTM based on attention mechanism[J]. Application Research of Computers, 2018, 35(5):1417-1420, 1440. (in Chinese)
[7] NIO L, SAKTI S, NEUBIG G, et al. Conversation dialog corpora from television and movie scripts[C]//2014 17th Oriental Chapter of the International Committee for the Co-ordination and Standardization of Speech Databases and Assessment Techniques (COCOSDA). Phuket, Thailand:IEEE Press, 2014:1-4.
[8] 曾小芹. 基于Python的中文结巴分词技术实现[J]. 信息与电脑(理论版), 2019, 31(18):38-39, 42. ZENG X Q. Technology implementation of Chinese jieba segmentation based on python[J]. China Computer & Communication, 2019, 31(18):38-39, 42. (in Chinese)
[9] RONG X. Word2vec parameter learning explained[EB/OL]. (2016-06-05)[2021-05-01]. https://arxiv.org/abs/1411.2738v4.
[10] 张伟男, 张杨子, 刘挺. 对话系统评价方法综述[J]. 中国科学:信息科学, 2017, 47(8):953-966. ZHANG W N, ZHANG Y Z, LIU T. Survey of evaluation methods for dialogue systems[J]. Scientia Sinica (Informationis), 2017, 47(8):953-966. (in Chinese)
[11] MOLDOVAN D I, TATU M. Natural language question answering system and method utilizing multi-modal logic:US20060053000 A1[P]. 2006-03-09.
[12] 邢超. 智能问答系统的设计与实现[D]. 北京:北京交通大学, 2015. XING C. The design and implementation of intelligent question and answering system[D]. Beijing:Beijing Jiaotong University, 2015. (in Chinese)
[13] WEN T H, VANDYKE D, MRKSIC N, et al. A network-based end-to-end trainable task-oriented dialogue system[EB/OL]. (2017-04-24)[2021-05-01]. https://arxiv.org/abs/1604.04562v3.
[14] 张杰晖. 任务型对话系统的自然语言生成研究[D]. 广州:华南理工大学, 2019. ZHANG J H. Research on natural language generation in task-based dialogue system[D]. Guangzhou:South China University of Technology, 2019. (in Chinese)
[15] MĚKOTA O, GÖKIRMAK M, LAITOCH P. End to end dialogue transformer[EB/OL]. (2020-08-24)[2021-05-01]. https://www.researchgate.net/publication/343849046_End _to_End_Dialogue_Transformer.
[16] THIERGART J, HUBER S, VBELLACKER T. Under-standing emails and drafting responses-An approach using GPT-3[EB/OL]. (2021-02-15)[2021-05-01]. https://arxiv.org/abs/2102.03062v3.
[17] 张献涛, 张猛, 暴筱, 等. 一种提升人机交互对话语料质量与多样性的对话语料库生成方法:CN111026884A[P]. 2020-04-17. ZHANG X T, ZHANG M, BAO X, et al. Dialogue corpus generation method for improving quality and diversity of man-machine interaction dialogue corpora:CN111026884A[P]. 2020-04-17. (in Chinese)
[18] WIKIPEDIA. Long short-term memory[EB/OL]. (2021-03-25)[2021-05-01]. https://en.wikipedia.org/wiki/Long_short-term_memory.
[19] WIKIPEDIA. Attention (machine learning)[EB/OL]. (2021-02-27)[2021-05-01]. https://en.wikipedia.org/wiki/Attention_(machine_learning).
[20] WIKIPEDIA. BLEU[EB/OL]. (2020-11-09)[2021-05-01]. https://en.wikipedia.org/wiki/BLEU.
[21] WIKIPEDIA. Autoencoder[EB/OL]. (2021-03-24)[2021-05-01]. https://en.wikipedia.org/wiki/Autoencoder.
[22] DANESCU-NICULESCU-MIZIL C, LEE L. Chameleons in imagined conversations:A new approach to understanding coordination of linguistic style in dialogs[C]//CMCL 2011:Proceedings of the 2nd Workshop on Cognitive Modeling and Computational Linguistics. Portland, Oregon, USA:Association for Computational Linguistics, 2011:76-87.
[23] GITHUB. Fxsjy/jieba[DB/OL]. (2020-01-20)[2021-05-01]. https://github.com/fxsjy/jieba.
[24] WIKIPEDIA. Beam search[EB/OL]. (2021-03-11)[2021-05-01]. https://en.wikipedia.org/wiki/Beam_search.
