Multi-neural network collaboration for Chinese military named entity recognition

YIN Xuezhen; ZHAO Hui; ZHAO Junbao; YAO Wanwei; HUANG Zelin

doi:10.16511/j.cnki.qhdxxb.2020.25.004

PDF(1044 KB)

Journal of Tsinghua University(Science and Technology) ›› 2020, Vol. 60 ›› Issue (8) : 648-655. DOI: 10.16511/j.cnki.qhdxxb.2020.25.004

SPECIALSECTION: DATABASE

Multi-neural network collaboration for Chinese military named entity recognition

{{article.zuoZhe_EN}}

Author information +

History +

Abstract

Web data contains a large amount of high-value military information which has become an important data source for open-source military intelligence. Military named entity recognition is a basic, key task for information extraction, question answering and knowledge graphs in the military domain. Military named entity recognition faces some unique challenges not seen in searches for named entities in other domains, such as military named entity boundaries being vague and difficult to define, lack of standardized military terms in Internet media, extensive use of abbreviations, and the lack of a public military-oriented corpus. This paper presents an entity labeling strategy that includes the effects of fuzzy entity boundaries and a military-oriented corpus called MilitaryCorpus based on microblog data constructed by combining domain expert knowledge. A multi-neural network collaboration approach is then developed based on a named entity recognition model. The character level features are learned in the BERT (bidirectional encoder representations from transformers)-based Chinese character embedding representation layer with the context features extracted in the BiLSTM (bi-directional long short-term memory) neural network layer to form the feature matrix. Finally, the optimal tag sequence is generated in the CRF (conditional random field) layer. Tests show that the recall rate and the F-score of the BERT-BiLSTM-CRF model are 28.48% and 18.65% higher than those of a CRF-based entity recognition model, 13.91% and 8.69% higher than those of a BiLSTM-CRF-based entity recognition model, and 7.08% and 5.15% higher than those of a CNN (convolutional neural networks)-BiLSTM-CRF-based model.

Key words

military named entity recognition / bidirectional encoder representations from transformers (BERT) / fuzzy boundary / multi-neural network

Cite this article

EndNote

Ris (Procite)

Bibtex

Download Citations

YIN Xuezhen, ZHAO Hui, ZHAO Junbao, YAO Wanwei, HUANG Zelin. Multi-neural network collaboration for Chinese military named entity recognition[J]. Journal of Tsinghua University(Science and Technology). 2020, 60(8): 648-655 https://doi.org/10.16511/j.cnki.qhdxxb.2020.25.004

References

[1] WILLIAMS H J, BLUM I. Defining second generation open source intelligence (OSINT) for the defense enterprise:RR-1964-OSD[R]. Santa Monica:RAND Corporation, 2018.
[2] LI J, SUN A X, HAN J L, et al. A survey on deep learning for named entity recognition[J]. arXiv preprint arXiv:1812.09449, 2018.
[3] YADAV V, BETHARD S. A survey on recent advances in named entity recognition from deep learning models[C]//Proceedings of the 27th International Conference on Computational Linguistics. Santa Fe, New Mexico, USA:Association for Computational Linguistics, 2018:2145-2158.
[4] HUANG Z H, XU W, YU K. Bidirectional LSTM-CRF models for sequence tagging[J]. arXiv preprint arXiv:1508.01991, 2015.
[5] LAMPLE G, BALLESTEROS M, SUBRAMANIAN S, et al. Neural architectures for named entity recognition[C]//Proceedings of NAACL-HLT 2016. San Diego, USA:Association for Computational Linguistics, 2016.
[6] CHIU J P C, NICHOLS E. Named entity recognition with bidirectional LSTM-CNNs[J]. Transactions of the Association for Computational Linguistics, 2016, 4:357-370.
[7] KURU O, CAN O A, YURET D. CharNER:Character-level named entity recognition[C]//Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics:Technical Papers. Osaka, Japan:The COLING 2016 Organizing Committee, 2016:911-921.
[8] WANG Y, XIA B, LIU Z, et al. Named entity recognition for Chinese telecommunications field based on Char2Vec and Bi-LSTMs[C]//2017 12th International Conference on Intelligent Systems and Knowledge Engineering (ISKE). Nanjing, China:IEEE, 2017:1-7.
[9] 冯蕴天, 张宏军, 郝文宁. 面向军事文本的命名实体识别[J]. 计算机科学, 2015, 42(7):15-18, 47. FENG Y T, ZHANG H J, HAO W N. Named entity recognition for military text[J]. Computer Science, 2015, 42(7):15-18, 47. (in Chinese)
[10] 游飞, 张激, 邱定, 等. 基于深度神经网络的武器名称识别[J]. 计算机系统应用, 2018, 27(1):239-243. YOU F, ZHANG J, QIU D, et al. Weapon named entity recognition based on deep neural network[J]. Computer Systems & Applications, 2018, 27(1):239-243. (in Chinese)
[11] 王学锋, 杨若鹏, 朱巍. 基于深度学习的军事命名实体识别方法[J]. 装甲兵工程学院学报, 2018, 32(4):94-98. WANG X F, YANG R P, ZHU W. Military named entity recognition method based on deep learning[J]. Journal of Academy of Armored Force Engineering, 2018, 32(4):94-98. (in Chinese)
[12] 张晓海, 操新文, 高源. 基于深度学习的作战文书命名实体识别[J]. 指挥控制与仿真, 2019, 41(4):22-26. ZHANG X H, CAO X W, GAO Y. Named entity recognition for combat documents based on deep learning[J]. Command Control & Simulation, 2019, 41(4):22-26. (in Chinese)
[13] 李丽双, 郭元凯. 基于CNN-BLSTM-CRF模型的生物医学命名实体识别[J]. 中文信息学报, 2018, 32(1):116-122.LI L S, GUO Y K. Biomedical named entity recognition with CNN-BLSTM-CRF[J]. Journal of Chinese Information Processing, 2018, 32(1):116-122. (in Chinese)
[14] WU Y H, JIANG M, LEI J B, et al. Named entity recognition in Chinese clinical text using deep neural network[J]. Studies in Health Technology and Informatics, 2015, 216:624-628.
[15] PENG N Y, DREDZE M. Named entity recognition for Chinese social media with jointly trained embeddings[C]//Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. Lisbon, Portugal:Association for Computational Linguistics, 2015:548-554.
[16] MIKOLOV T, CHEN K, CORRADO G, et al. Efficient estimation of word representations in vector space[J]. Computer Science, arXiv preprint arXiv:1301. 3781, 2013.
[17] PENNINGTON J, SOCHER R, MANNING C D. Glove:Global vectors for word representation[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). Doha, Qatar:Association for Computational Linguistics, 2014:1532-1543.
[18] DEVLIN J, CHANG M W, LEE K, et al. Bert:Pre-training of deep bidirectional transformers for language understanding[J]. arXiv preprint arXiv:1810.04805, 2018.
[19] CHEN Q, ZHUO Z, WANG W. BERT for joint intent classification and slot filling[J]. arXiv preprint arXiv:1902.10909, 2019.
[20] ADHIKARI A, RAM A, TANG R, et al. DocBERT:BERT for document classification[J]. arXiv preprint arXiv:1904.08398, 2019.
[21] ALBERTI C, LEE K, COLLINS M. A BERT baseline for the natural questions[J]. arXiv preprint arXiv:1901.08634, 2019.