RI-TSP: Reasoning implicit sentiment with teacher-student prompting

Haiyan WU, Xiaojiang YU, Chaoqun SUN, Chengxiong LU, Yong DING, Di ZHOU, Shengchun DENG

Journal of Tsinghua University(Science and Technology) ›› 2025, Vol. 65 ›› Issue (8) : 1530-1540.

PDF(5695 KB)
PDF(5695 KB)
Journal of Tsinghua University(Science and Technology) ›› 2025, Vol. 65 ›› Issue (8) : 1530-1540. DOI: 10.16511/j.cnki.qhdxxb.2025.27.022
Computer Science and Technology

RI-TSP: Reasoning implicit sentiment with teacher-student prompting

Author information +
History +

Abstract

Objective: Sentiment analysis is an important subtask of natural language processing. Its core is to mine the sentiment aspects and sentiment polarity pairs in sentences and conduct relevant analysis. It is mainly divided into explicit sentiment analysis (ESA) and implicit sentiment analysis (ISA). Unlike the former, the latter mainly analyzes the implicit semantics of sentences that do not explicitly contain emotional polarity but clearly express emotional views in the context. ISA is widely used in fine-grained sentiment analysis. Currently, ESA dominates the sentiment analysis tasks, and its comment sentences explicitly contain sentiment partial words and sentiment polarity. In the existing work, text representations are mainly obtained through pretrained language models and emotional semantic information is inferred through fine-tuning; however, the deeper implicit semantic reasoning analysis of text still has shortcomings, and the fusion produces an "illusion." Methods: Therefore, this study first proposes a framework for implicit sentiment reasoning based on teacher-student prompting (reasoning implicit sentiment with teacher-student prompting, RI-TSP). Through the reasoning of three-layer thought chains, the implicit emotional information in the sentence can be more effectively mined. Second, a prompt fine-tuning paradigm was designed from the teacher to the student model. In this paradigm, the teacher model uses the zero-shot method to generate reasoning samples, and then, the student model performs prompt fine-tuning and training. Finally, the fine-tuning training of the large model is transferred to the small-scale model through the knowledge distillation method. Results: Experimental results show that on the Laptops and Restaurants datasets, the proposed RI-TSP method outperformed state-of-the-art methods, improving implicit sentiment inference accuracy by 1.73% and 3.49%, F1 value by 2.30% and 1.46%, and training efficiency by 25.0% and 50.0%, respectively. For these two datasets, the RI-TSP model achieved a higher improvement in ESA compared with RGAT, BERT+SPC, BERT+ADA, BERT+RGAT, and prompt-based models. Conclusions: Using a large-scale teacher language model, a sentiment polarity prompt method was developed to generate reasoning samples. Using the knowledge of thought chain and prompt learning, samples with thought chain reasoning processes were effectively generated. In addition, through knowledge distillation, samples with reasoning knowledge were fine-tuned on a small-scale student model to achieve sentiment polarity reasoning. Experimental results on the public datasets Laptops and Restaurants showed that the RI-TSP model had a high accuracy rate and low running cost.

Key words

large model / transfer learning / prompt tuning / chain of thought reasoning / sentiment analysis

Cite this article

Download Citations
Haiyan WU , Xiaojiang YU , Chaoqun SUN , et al . RI-TSP: Reasoning implicit sentiment with teacher-student prompting[J]. Journal of Tsinghua University(Science and Technology). 2025, 65(8): 1530-1540 https://doi.org/10.16511/j.cnki.qhdxxb.2025.27.022

References

1
WANG Y Q, HUANG M L, ZHU X Y, et al. Attention-based LSTM for aspect-level sentiment classification[C]//Proceedings of 2016 Conference on Empirical Methods in Natural Language Processing. Austin, Texas: Association for Computational Linguistics, 2016: 606-615.
2
TANG D Y, QIN B, FENG X C, et al. Effective LSTMs for target-dependent sentiment classification[C]//Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers. Osaka, Japan: The COLING 2016 Organizing Committee, 2016: 3298-3307.
3
CHEN P, SUN Z Q, BING L D, et al. Recurrent attention network on memory for aspect sentiment analysis[C]//Proceedings of 2017 Conference on Empirical Methods in Natural Language Processing. Copenhagen, Denmark: Association for Computational Linguistics, 2017: 452-461.
4
HOANG M, BIHORAC O A, ROUCES J. Aspect-based sentiment analysis using BERT[C]//Proceedings of the 22nd Nordic Conference on Computational Linguistics. Turku, Finland: Linköping University Electronic Press, 2019: 187-196.
5
NGUYEN T H, SHIRAI K. PhraseRNN: Phrase recursive neural network for aspect-based sentiment analysis[C]//Proceedings of 2015 Conference on Empirical Methods in Natural Language Processing. Lisbon, Portugal: Association for Computational Linguistics, 2015: 2509-2514.
6
LIU P F, QIU X P, HUANG X J. Adaptive semantic compositionality for sentence modelling[C]// Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence. Melbourne, Australia: Morgan Kaufmann. 2017: 4061-4067.
7
SUN K, ZHANG R C, MENSAH S, et al. Aspect-level sentiment analysis via convolution over dependency tree[C]//Proceedings of 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). Hong Kong, China: Association for Computational Linguistics, 2019: 5679-5688.
8
ZHANG C, LI Q C, SONG D W. Aspect-based sentiment classification with aspect-specific graph convolutional networks[C]//Proceedings of 2019 Conference on Empirical Methods in Natural Language Processing. Hong Kong, China: Association for Computational Linguistics, 2019: 4568-4578.
9
HUANG B X, CARLEY K. Syntax-aware aspect level sentiment classification with graph attention networks[C]//Proceedings of 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). Hong Kong, China: Association for Computational Linguistics, 2019: 5469-5477.
10
WANG K, SHEN W Z, YANG Y Y, et al. Relational graph attention network for aspect-based sentiment analysis[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Online: Association for Computational Linguistics, 2020: 3229-3238.
11
TANG H, JI D H, LI C L, et al. Dependency graph enhanced dual-transformer structure for aspect-based sentiment classification[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Online: Association for Computational Linguistics, 2020: 6578-6588.
12
LI R, CHEN H, FENG F, et al. Dual graph convolutional networks for aspect-based sentiment analysis[C]//Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). Online: Association for Computational Linguistics, 2021: 6319-6329.
13
CHEN C H, TENG Z Y, WANG Z Q, et al. Discrete opinion tree induction for aspect-based sentiment analysis[C]//Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Dublin, Ireland: Association for Computational Linguistics, 2022: 2051-2064.
14
FEI H, LI B B, LIU Q, et al. Reasoning implicit sentiment with chain-of-thought prompting[C]//Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). Toronto, Canada: Association for Computational Linguistics, 2023: 1171-1182.
15
CHOWDHERY A , NARANG S , DEVLIN J , et al. Palm: Scaling language modeling with pathways[J]. Journal of Machine Learning Research, 2023, 24 (240): 1- 113.
16
WEI J, WANG X Z, SCHUURMANS D, et al. Chain-of-thought prompting elicits reasoning in large language models[C]//Proceedings of the 36th International Conference on Neural Information Processing Systems. New Orleans, USA: Curran Associates Inc., 2022: 1800.
17
KOJIMA T, GU S S, REID M, et al. Large language models are zero-shot reasoners[C]//Proceedings of the 36th International Conference on Neural Information Processing Systems. New Orleans, LA, USA: Curran Associates Inc., 2022: 1613.
18
GAO L Y, MADAAN A, ZHOU S Y, et al. PAL: Program-aided language models[C]//Proceedings of the 40th International Conference on Machine Learning. Honolulu, HI, USA: JMLR. org, 2023: 435.
19
ZHEN B, ZHANG N Y, JIANG Y N, et al. When do program-of-thought works for reasoning?[C]//Proceedings of the Thirty-Eighth AAAI Conference on Artificial Intelligence. Vancouver, Canada: AAAI Press. 2024: 17691-17699.
20
ZHANG T H, GE J X, LUO H, Y et al. Natural language embedded programs for hybrid language symbolic reasoning[C]//Proceedings of the Findings of the Association for Computational Linguistics: NAACL 2024. Mexico City, Mexico: Association for Computational Linguistics, 2024: 4131-4155.
21
IMANI S, DU L, SHRIVASTAVA H. MathPrompter: Mathematical reasoning using large language models[C]//Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 5): Industry Track. Toronto, Canada: Association for Computational Linguistics, 2023: 37-42.
22
ZHANG Z S, ZHANG A, LI M, et al. Automatic chain of thought prompting in large language models[C]//Proceedings of the 11th International Conference on Learning Representations. Kigali, Rwanda: ICLR, 2023: 1-32.
23
WAN X C, SUN R X, DAI H J, et al. Better zero-Shot reasoning with self-adaptive prompting[C]//Proceedings of the Findings of the Association for Computational Linguistics: ACL 2023. Toronto, Canada: Association for Computational Linguistics, 2023: 3493-3514.
24
WANG X Z, WEI J, SCHUURMANS D, et al. Self-consistency improves chain of thought reasoning in language models[C]//Proceedings of the 11th International Conference on Learning Representations. Kigali, Rwanda: ICLR, 2023: 1-24.
25
WANG L, XU W Y, LAN Y H, et al. Plan-and-solve prompting: Improving zero-shot chain-of-thought reasoning by large language models[C]//Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Toronto, Canada: Association for Computational Linguistics, 2023: 2609-2634.
26
SHAO Z H, GONG Y Y, SHEN Y L, et al. Synthetic prompting: Generating chain-of-thought demonstrations for large language models[C]//Proceedings of the 40th International Conference on Machine Learning. Honolulu, HI, USA: JMLR. org, 2023: 1273.
27
HU H X, LU H Y, ZHANG H J, et al. Chain-of-symbol prompting for spatial reasoning in large language models[C]//First Conference on Language Modeling. Philadelphia, PA, USA: COLM Organization, 2024: 1434.
28
RANALDI L, PUCCI G, RANALDI F, et al. A tree-of-thoughts to broaden multi-step reasoning across languages[C]//Findings of the Association for Computational Linguistics. Mexico City, Mexico: Association for Computational Linguistics. 2024: 1229-1241.
29
YAO S Y, YU D, ZHAO J, et al. Tree of thoughts: Deliberate problem solving with large language models[C]//Proceedings of the 37th International Conference on Neural Information Processing Systems. New Orleans, LA, USA: Curran Associates Inc., 2024: 517.
30
MO S T, XIN M. Tree of uncertain thoughts reasoning for large language models[C]//Proceedings of the ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Seoul, Korea, Republic of: IEEE, 2024: 12742-12746.
31
BESTA M, BLACH N, KUBICEK A, et al. Graph of thoughts: Solving elaborate problems with large language models[C]//Proceedings of the Thirty-Eighth AAAI Conference on Artificial Intelligence. Vancouver, Canada: AAAI Press, 2024: 17682-17690.
32
GOU J P , YU B S , MAYBANK S J , et al. Knowledge distillation: A survey[J]. International Journal of Computer Vision, 2021, 129 (6): 1789- 1819.
33
ZELIKMAN E, WU Y H, MU J, et al. STaR: Bootstrapping reasoning with reasoning[C]//Proceedings of the 36th International Conference on Neural Information Processing Systems. New Orleans, LA, USA: Curran Associates, Inc., 2022: 1126.
34
LI L H, HESSEL J, YU Y, et al. Symbolic chain-of-thought distillation: Small models can also "think" step-by-step[C]//Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Toronto, Canada: Association for Computational Linguistics, 2023: 2665-2679.
35
WANG P, WANG Z, LI Z, et al. SCOTT: Self-consistent chain-of-thought distillation[C]//Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Toronto, Canada: ACL 2023: 5546-5558.
36
TIAN Y H, CHEN G M, SONG Y. Aspect-based sentiment analysis with type-aware graph convolutional networks and layer ensemble[C]//Proceedings of 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Online: Association for Computational Linguistics, 2021: 2910-2922.
37
CAI H J, TU Y F, ZHOU X S, et al. Aspect-category based sentiment analysis with hierarchical graph convolutional network[C]//Proceedings of the 28th International Conference on Computational Linguistics. Barcelona, Spain: International Committee on Computational Linguistics, 2020: 833-843.
38
PANG S G, YAN Z H, HUANG W H, et al. Highway-based local graph convolution network for aspect based sentiment analysis[C]//Proceedings of the 10th CCF International Conference ON Natural Language Processing and Chinese Computing. Qingdao, China: Springer, 2021: 544-556.
39
WU H Y , ZHANG Z Q , SHI S Y , et al. Phrase dependency relational graph attention network for aspect-based sentiment analysis[J]. Knowledge-Based Systems, 2022, 236, 107736.
40
鲍小异, 姜晓彤, 王中卿, 等. 基于跨语言图神经网络模型的属性级情感分类[J]. 软件学报, 2023, 34 (2): 676- 689.
BAO X Y , JIANG X T , WANG Z Q , et al. Cross-lingual aspect-level sentiment classification with graph neural network[J]. Journal of Software, 2023, 34 (2): 676- 689.
41
郭贤伟, 赖华, 余正涛, 等. 融合情绪知识的案件微博评论情绪分类[J]. 计算机学报, 2021, 44 (3): 564- 578.
GUO X W , LAI H , YU Z T , et al. Emotion classification of case-related microblog comments integrating emotional knowledge[J]. Chinese Journal of Computers, 2021, 44 (3): 564- 578.
42
BROWN T B, MANN B, RYDER N, et al. Language models are few-shot learners[C]// Proceedings of the 34th International Conference on Neural Information Processing Systems. Vancouver, Canada: Curran Associates Inc., 2020: 159.
43
SCHICK T, SCHÜTZE H. Few-Shot text generation with natural language instructions[C]//Proceedings of 2021 Conference on Empirical Methods in Natural Language Processing. Punta Cana, Dominican Republic: Association Computational Linguistics, 2021: 390-402.
44
SCHICK T, SCHÜTZE H. It's not just size that matters: Small language models are also few-shot learners[C]//Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Online: Association for Computational Linguistics, 2021: 2339-2352.
45
SHIN R, LIN C, THOMSON S, et al. Constrained language models yield few-shot semantic parsers[C]//Proceedings of 2021 Conference on Empirical Methods in Natural Language Processing. Punta Cana, Dominican Republic: Association for Computational Linguistics, 2021: 7699-7715.
46
饶元, 吴连伟, 王一鸣, 等. 基于语义分析的情感计算技术研究进展[J]. 软件学报, 2018, 29 (8): 2397- 2426.
RAO Y , WU L W , WANG Y M , et al. Research progress on emotional computation technology based on semantic analysis[J]. Journal of Software, 2018, 29 (8): 2397- 2426.
47
JIANG Z B, XU F F, ARAKI J, et al. How can we know what language models know?[M]//JOHNSON M, ROARK B, NENKOVA A. Transactions of the Association for Computational Linguistics. Cambridge: MIT Press, 2020: 423-438.
48
SHIN T, RAZEGHI Y, LOGAN IV R L, et al. AutoPrompt: Eliciting knowledge from language models with automatically generated prompts[C]//Proceedings of 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). Online: Association for Computational Linguistics, 2020: 4222-4235.
49
HO N, SCHMID L, YUN S Y. Large language models are reasoning teachers[C]//Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Toronto, Canada: ACL, 2023: 14852-14882.
50
MAGISTER L C, MALLINSON J, ADAMEK J, et al. Teaching small language models to reason[C]//Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). Toronto, Canada: ACL, 2023: 1773-1781.
51
LIU X, JI K X, FU Y C, et al. P-tuning v2: Prompt tuning can be comparable to fine-tuning universally across scales and tasks[C]//Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). Dublin, Ireland: Association for Computational Linguistics, 2022: 61-68.
52
LI Z Y, ZOU Y C, ZHANG C, et al. Learning implicit sentiment in aspect-based sentiment analysis with supervised contrastive pre-training[C]//Proceedings of 2021 Conference on Empirical Methods in Natural Language Processing. Punta Cana, Dominican Republic: Association for Computational Linguistics, 2021: 246-256.
53
DEVLIN J, CHANG M W, LEE K, et al. BERT: Pre-training of deep bidirectional transformers for language understanding[C]//Proceedings of 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). Minneapolis, Minnesota: Association for Computational Linguistics, 2019: 4171-4186.
54
RIETZLER A, STABINGER S, OPITZ P, et al. Adapt or get left behind: Domain adaptation through BERT language model finetuning for aspect-target sentiment classification[C]// Proceedings of the Twelfth Language Resources and Evaluation Conference. Marseille, France: European Language Resources Association, 2020: 4933-4941.

RIGHTS & PERMISSIONS

All rights reserved. Unauthorized reproduction is prohibited.
PDF(5695 KB)

Accesses

Citation

Detail

Sections
Recommended

/