基于教师-学生模型的隐式情感推理

吴海燕, 虞小江, 孙超群, 卢成雄, 丁勇, 周迪, 邓胜春

清华大学学报(自然科学版) ›› 2025, Vol. 65 ›› Issue (8) : 1530-1540.

PDF(5695 KB)
PDF(5695 KB)
清华大学学报(自然科学版) ›› 2025, Vol. 65 ›› Issue (8) : 1530-1540. DOI: 10.16511/j.cnki.qhdxxb.2025.27.022
计算机科学与技术

基于教师-学生模型的隐式情感推理

作者信息 +

RI-TSP: Reasoning implicit sentiment with teacher-student prompting

Author information +
文章历史 +

摘要

与显式情感分析不同的是,隐式情感分析主要是针对那些不显式包含情感极性却在语境中明确表达了情感观点的句子进行的深层次隐含语义分析,其在细粒度情感分析中被广泛应用。目前已有工作主要通过预训练语言模型来获取文本表示并通过微调推理情感语义信息,但对于文本深层次的隐含语义推理分析仍有不足,容易产生“幻觉”现象。该文提出一种基于教师-学生模型的隐式情感推理框架(reasoning implicit sentiment with teacher-student prompting, RI-TSP),其中教师模型采用大规模语言模型T5,运用零样本思维链提示构建推理样本,并利用知识蒸馏、迁移学习等技术使训练小模型具备一定的语义分析和多跳推理能力。实验结果表明,在Laptops和Restaurants这2个公开数据集上,该文提出的RI-TSP方法与基于Prompt的模型相比,在隐式情感推理分析中的准确率分别提升了1.73%和3.49%,F1值分别提升了2.30%和1.46%,训练效率分别提升了25.0%和50.0%。

Abstract

Objective: Sentiment analysis is an important subtask of natural language processing. Its core is to mine the sentiment aspects and sentiment polarity pairs in sentences and conduct relevant analysis. It is mainly divided into explicit sentiment analysis (ESA) and implicit sentiment analysis (ISA). Unlike the former, the latter mainly analyzes the implicit semantics of sentences that do not explicitly contain emotional polarity but clearly express emotional views in the context. ISA is widely used in fine-grained sentiment analysis. Currently, ESA dominates the sentiment analysis tasks, and its comment sentences explicitly contain sentiment partial words and sentiment polarity. In the existing work, text representations are mainly obtained through pretrained language models and emotional semantic information is inferred through fine-tuning; however, the deeper implicit semantic reasoning analysis of text still has shortcomings, and the fusion produces an "illusion." Methods: Therefore, this study first proposes a framework for implicit sentiment reasoning based on teacher-student prompting (reasoning implicit sentiment with teacher-student prompting, RI-TSP). Through the reasoning of three-layer thought chains, the implicit emotional information in the sentence can be more effectively mined. Second, a prompt fine-tuning paradigm was designed from the teacher to the student model. In this paradigm, the teacher model uses the zero-shot method to generate reasoning samples, and then, the student model performs prompt fine-tuning and training. Finally, the fine-tuning training of the large model is transferred to the small-scale model through the knowledge distillation method. Results: Experimental results show that on the Laptops and Restaurants datasets, the proposed RI-TSP method outperformed state-of-the-art methods, improving implicit sentiment inference accuracy by 1.73% and 3.49%, F1 value by 2.30% and 1.46%, and training efficiency by 25.0% and 50.0%, respectively. For these two datasets, the RI-TSP model achieved a higher improvement in ESA compared with RGAT, BERT+SPC, BERT+ADA, BERT+RGAT, and prompt-based models. Conclusions: Using a large-scale teacher language model, a sentiment polarity prompt method was developed to generate reasoning samples. Using the knowledge of thought chain and prompt learning, samples with thought chain reasoning processes were effectively generated. In addition, through knowledge distillation, samples with reasoning knowledge were fine-tuned on a small-scale student model to achieve sentiment polarity reasoning. Experimental results on the public datasets Laptops and Restaurants showed that the RI-TSP model had a high accuracy rate and low running cost.

关键词

大模型 / 迁移学习 / 提示微调 / 思维链推理 / 情感分析

Key words

large model / transfer learning / prompt tuning / chain of thought reasoning / sentiment analysis

引用本文

导出引用
吴海燕, 虞小江, 孙超群, . 基于教师-学生模型的隐式情感推理[J]. 清华大学学报(自然科学版). 2025, 65(8): 1530-1540 https://doi.org/10.16511/j.cnki.qhdxxb.2025.27.022
Haiyan WU, Xiaojiang YU, Chaoqun SUN, et al. RI-TSP: Reasoning implicit sentiment with teacher-student prompting[J]. Journal of Tsinghua University(Science and Technology). 2025, 65(8): 1530-1540 https://doi.org/10.16511/j.cnki.qhdxxb.2025.27.022
中图分类号: TP18   

参考文献

1
WANG Y Q, HUANG M L, ZHU X Y, et al. Attention-based LSTM for aspect-level sentiment classification[C]//Proceedings of 2016 Conference on Empirical Methods in Natural Language Processing. Austin, Texas: Association for Computational Linguistics, 2016: 606-615.
2
TANG D Y, QIN B, FENG X C, et al. Effective LSTMs for target-dependent sentiment classification[C]//Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers. Osaka, Japan: The COLING 2016 Organizing Committee, 2016: 3298-3307.
3
CHEN P, SUN Z Q, BING L D, et al. Recurrent attention network on memory for aspect sentiment analysis[C]//Proceedings of 2017 Conference on Empirical Methods in Natural Language Processing. Copenhagen, Denmark: Association for Computational Linguistics, 2017: 452-461.
4
HOANG M, BIHORAC O A, ROUCES J. Aspect-based sentiment analysis using BERT[C]//Proceedings of the 22nd Nordic Conference on Computational Linguistics. Turku, Finland: Linköping University Electronic Press, 2019: 187-196.
5
NGUYEN T H, SHIRAI K. PhraseRNN: Phrase recursive neural network for aspect-based sentiment analysis[C]//Proceedings of 2015 Conference on Empirical Methods in Natural Language Processing. Lisbon, Portugal: Association for Computational Linguistics, 2015: 2509-2514.
6
LIU P F, QIU X P, HUANG X J. Adaptive semantic compositionality for sentence modelling[C]// Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence. Melbourne, Australia: Morgan Kaufmann. 2017: 4061-4067.
7
SUN K, ZHANG R C, MENSAH S, et al. Aspect-level sentiment analysis via convolution over dependency tree[C]//Proceedings of 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). Hong Kong, China: Association for Computational Linguistics, 2019: 5679-5688.
8
ZHANG C, LI Q C, SONG D W. Aspect-based sentiment classification with aspect-specific graph convolutional networks[C]//Proceedings of 2019 Conference on Empirical Methods in Natural Language Processing. Hong Kong, China: Association for Computational Linguistics, 2019: 4568-4578.
9
HUANG B X, CARLEY K. Syntax-aware aspect level sentiment classification with graph attention networks[C]//Proceedings of 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). Hong Kong, China: Association for Computational Linguistics, 2019: 5469-5477.
10
WANG K, SHEN W Z, YANG Y Y, et al. Relational graph attention network for aspect-based sentiment analysis[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Online: Association for Computational Linguistics, 2020: 3229-3238.
11
TANG H, JI D H, LI C L, et al. Dependency graph enhanced dual-transformer structure for aspect-based sentiment classification[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Online: Association for Computational Linguistics, 2020: 6578-6588.
12
LI R, CHEN H, FENG F, et al. Dual graph convolutional networks for aspect-based sentiment analysis[C]//Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). Online: Association for Computational Linguistics, 2021: 6319-6329.
13
CHEN C H, TENG Z Y, WANG Z Q, et al. Discrete opinion tree induction for aspect-based sentiment analysis[C]//Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Dublin, Ireland: Association for Computational Linguistics, 2022: 2051-2064.
14
FEI H, LI B B, LIU Q, et al. Reasoning implicit sentiment with chain-of-thought prompting[C]//Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). Toronto, Canada: Association for Computational Linguistics, 2023: 1171-1182.
15
CHOWDHERY A , NARANG S , DEVLIN J , et al. Palm: Scaling language modeling with pathways[J]. Journal of Machine Learning Research, 2023, 24 (240): 1- 113.
16
WEI J, WANG X Z, SCHUURMANS D, et al. Chain-of-thought prompting elicits reasoning in large language models[C]//Proceedings of the 36th International Conference on Neural Information Processing Systems. New Orleans, USA: Curran Associates Inc., 2022: 1800.
17
KOJIMA T, GU S S, REID M, et al. Large language models are zero-shot reasoners[C]//Proceedings of the 36th International Conference on Neural Information Processing Systems. New Orleans, LA, USA: Curran Associates Inc., 2022: 1613.
18
GAO L Y, MADAAN A, ZHOU S Y, et al. PAL: Program-aided language models[C]//Proceedings of the 40th International Conference on Machine Learning. Honolulu, HI, USA: JMLR. org, 2023: 435.
19
ZHEN B, ZHANG N Y, JIANG Y N, et al. When do program-of-thought works for reasoning?[C]//Proceedings of the Thirty-Eighth AAAI Conference on Artificial Intelligence. Vancouver, Canada: AAAI Press. 2024: 17691-17699.
20
ZHANG T H, GE J X, LUO H, Y et al. Natural language embedded programs for hybrid language symbolic reasoning[C]//Proceedings of the Findings of the Association for Computational Linguistics: NAACL 2024. Mexico City, Mexico: Association for Computational Linguistics, 2024: 4131-4155.
21
IMANI S, DU L, SHRIVASTAVA H. MathPrompter: Mathematical reasoning using large language models[C]//Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 5): Industry Track. Toronto, Canada: Association for Computational Linguistics, 2023: 37-42.
22
ZHANG Z S, ZHANG A, LI M, et al. Automatic chain of thought prompting in large language models[C]//Proceedings of the 11th International Conference on Learning Representations. Kigali, Rwanda: ICLR, 2023: 1-32.
23
WAN X C, SUN R X, DAI H J, et al. Better zero-Shot reasoning with self-adaptive prompting[C]//Proceedings of the Findings of the Association for Computational Linguistics: ACL 2023. Toronto, Canada: Association for Computational Linguistics, 2023: 3493-3514.
24
WANG X Z, WEI J, SCHUURMANS D, et al. Self-consistency improves chain of thought reasoning in language models[C]//Proceedings of the 11th International Conference on Learning Representations. Kigali, Rwanda: ICLR, 2023: 1-24.
25
WANG L, XU W Y, LAN Y H, et al. Plan-and-solve prompting: Improving zero-shot chain-of-thought reasoning by large language models[C]//Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Toronto, Canada: Association for Computational Linguistics, 2023: 2609-2634.
26
SHAO Z H, GONG Y Y, SHEN Y L, et al. Synthetic prompting: Generating chain-of-thought demonstrations for large language models[C]//Proceedings of the 40th International Conference on Machine Learning. Honolulu, HI, USA: JMLR. org, 2023: 1273.
27
HU H X, LU H Y, ZHANG H J, et al. Chain-of-symbol prompting for spatial reasoning in large language models[C]//First Conference on Language Modeling. Philadelphia, PA, USA: COLM Organization, 2024: 1434.
28
RANALDI L, PUCCI G, RANALDI F, et al. A tree-of-thoughts to broaden multi-step reasoning across languages[C]//Findings of the Association for Computational Linguistics. Mexico City, Mexico: Association for Computational Linguistics. 2024: 1229-1241.
29
YAO S Y, YU D, ZHAO J, et al. Tree of thoughts: Deliberate problem solving with large language models[C]//Proceedings of the 37th International Conference on Neural Information Processing Systems. New Orleans, LA, USA: Curran Associates Inc., 2024: 517.
30
MO S T, XIN M. Tree of uncertain thoughts reasoning for large language models[C]//Proceedings of the ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Seoul, Korea, Republic of: IEEE, 2024: 12742-12746.
31
BESTA M, BLACH N, KUBICEK A, et al. Graph of thoughts: Solving elaborate problems with large language models[C]//Proceedings of the Thirty-Eighth AAAI Conference on Artificial Intelligence. Vancouver, Canada: AAAI Press, 2024: 17682-17690.
32
GOU J P , YU B S , MAYBANK S J , et al. Knowledge distillation: A survey[J]. International Journal of Computer Vision, 2021, 129 (6): 1789- 1819.
33
ZELIKMAN E, WU Y H, MU J, et al. STaR: Bootstrapping reasoning with reasoning[C]//Proceedings of the 36th International Conference on Neural Information Processing Systems. New Orleans, LA, USA: Curran Associates, Inc., 2022: 1126.
34
LI L H, HESSEL J, YU Y, et al. Symbolic chain-of-thought distillation: Small models can also "think" step-by-step[C]//Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Toronto, Canada: Association for Computational Linguistics, 2023: 2665-2679.
35
WANG P, WANG Z, LI Z, et al. SCOTT: Self-consistent chain-of-thought distillation[C]//Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Toronto, Canada: ACL 2023: 5546-5558.
36
TIAN Y H, CHEN G M, SONG Y. Aspect-based sentiment analysis with type-aware graph convolutional networks and layer ensemble[C]//Proceedings of 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Online: Association for Computational Linguistics, 2021: 2910-2922.
37
CAI H J, TU Y F, ZHOU X S, et al. Aspect-category based sentiment analysis with hierarchical graph convolutional network[C]//Proceedings of the 28th International Conference on Computational Linguistics. Barcelona, Spain: International Committee on Computational Linguistics, 2020: 833-843.
38
PANG S G, YAN Z H, HUANG W H, et al. Highway-based local graph convolution network for aspect based sentiment analysis[C]//Proceedings of the 10th CCF International Conference ON Natural Language Processing and Chinese Computing. Qingdao, China: Springer, 2021: 544-556.
39
WU H Y , ZHANG Z Q , SHI S Y , et al. Phrase dependency relational graph attention network for aspect-based sentiment analysis[J]. Knowledge-Based Systems, 2022, 236, 107736.
40
鲍小异, 姜晓彤, 王中卿, 等. 基于跨语言图神经网络模型的属性级情感分类[J]. 软件学报, 2023, 34 (2): 676- 689.
BAO X Y , JIANG X T , WANG Z Q , et al. Cross-lingual aspect-level sentiment classification with graph neural network[J]. Journal of Software, 2023, 34 (2): 676- 689.
41
郭贤伟, 赖华, 余正涛, 等. 融合情绪知识的案件微博评论情绪分类[J]. 计算机学报, 2021, 44 (3): 564- 578.
GUO X W , LAI H , YU Z T , et al. Emotion classification of case-related microblog comments integrating emotional knowledge[J]. Chinese Journal of Computers, 2021, 44 (3): 564- 578.
42
BROWN T B, MANN B, RYDER N, et al. Language models are few-shot learners[C]// Proceedings of the 34th International Conference on Neural Information Processing Systems. Vancouver, Canada: Curran Associates Inc., 2020: 159.
43
SCHICK T, SCHÜTZE H. Few-Shot text generation with natural language instructions[C]//Proceedings of 2021 Conference on Empirical Methods in Natural Language Processing. Punta Cana, Dominican Republic: Association Computational Linguistics, 2021: 390-402.
44
SCHICK T, SCHÜTZE H. It's not just size that matters: Small language models are also few-shot learners[C]//Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Online: Association for Computational Linguistics, 2021: 2339-2352.
45
SHIN R, LIN C, THOMSON S, et al. Constrained language models yield few-shot semantic parsers[C]//Proceedings of 2021 Conference on Empirical Methods in Natural Language Processing. Punta Cana, Dominican Republic: Association for Computational Linguistics, 2021: 7699-7715.
46
饶元, 吴连伟, 王一鸣, 等. 基于语义分析的情感计算技术研究进展[J]. 软件学报, 2018, 29 (8): 2397- 2426.
RAO Y , WU L W , WANG Y M , et al. Research progress on emotional computation technology based on semantic analysis[J]. Journal of Software, 2018, 29 (8): 2397- 2426.
47
JIANG Z B, XU F F, ARAKI J, et al. How can we know what language models know?[M]//JOHNSON M, ROARK B, NENKOVA A. Transactions of the Association for Computational Linguistics. Cambridge: MIT Press, 2020: 423-438.
48
SHIN T, RAZEGHI Y, LOGAN IV R L, et al. AutoPrompt: Eliciting knowledge from language models with automatically generated prompts[C]//Proceedings of 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). Online: Association for Computational Linguistics, 2020: 4222-4235.
49
HO N, SCHMID L, YUN S Y. Large language models are reasoning teachers[C]//Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Toronto, Canada: ACL, 2023: 14852-14882.
50
MAGISTER L C, MALLINSON J, ADAMEK J, et al. Teaching small language models to reason[C]//Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). Toronto, Canada: ACL, 2023: 1773-1781.
51
LIU X, JI K X, FU Y C, et al. P-tuning v2: Prompt tuning can be comparable to fine-tuning universally across scales and tasks[C]//Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). Dublin, Ireland: Association for Computational Linguistics, 2022: 61-68.
52
LI Z Y, ZOU Y C, ZHANG C, et al. Learning implicit sentiment in aspect-based sentiment analysis with supervised contrastive pre-training[C]//Proceedings of 2021 Conference on Empirical Methods in Natural Language Processing. Punta Cana, Dominican Republic: Association for Computational Linguistics, 2021: 246-256.
53
DEVLIN J, CHANG M W, LEE K, et al. BERT: Pre-training of deep bidirectional transformers for language understanding[C]//Proceedings of 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). Minneapolis, Minnesota: Association for Computational Linguistics, 2019: 4171-4186.
54
RIETZLER A, STABINGER S, OPITZ P, et al. Adapt or get left behind: Domain adaptation through BERT language model finetuning for aspect-target sentiment classification[C]// Proceedings of the Twelfth Language Resources and Evaluation Conference. Marseille, France: European Language Resources Association, 2020: 4933-4941.

基金

国家自然科学基金青年科学基金项目(62306267)
浙江省自然科学基金一般项目(LY22F020027)
浙江省普通本科高校“十四五”教学改革项目(jg20220383)
中央引导地方科技发展资金项目(2023ZY1069)

版权

版权所有,未经授权,不得转载。
PDF(5695 KB)

Accesses

Citation

Detail

段落导航
相关文章

/