融合语义角色和自注意力机制的中文文本蕴含识别
A Chinese Textual Entailment Recognition Method Incorporating Semantic Role and Self-Attention
查看参考文献24篇
|
文摘
|
文本蕴含识别旨在识别两个给定句子之间的逻辑关系.本文通过构造语义角色和自注意力机制融合模块,把句子的深层语义信息与Transformer模型的编码部分相结合,从而增强自注意力机制捕获句子语义的能力.针对中文文本蕴含识别在数据集上存在规模小和噪声大的问题,使用大规模预训练语言模型能够提升模型在小规模数据集上的识别性能.实验结果表明,提出的方法在第十七届中国计算语言学大会中文文本蕴含识别评测数据集CNLI上的准确率达到了80. 28%. |
|
其他语种文摘
|
Recognizing textual entailment is intended to infer the logical relationship between two given sentences. In this paper,we incorporate the deep semantic information of sentences and the encoder of Transformer by constructing the SRL-Attention fusion module, and it effectively improves the ability of self-attention mechanism to capture sentence semantics. Furthermore, concerning the small scale and high noise problems on the dataset,we use large-scale pre-trained language model improving the recognition performance of model on small-scale dataset. Experimental results show that the accuracy of our model on the dataset CNLI, it is released as Chinese textual entailment recognition evaluation corpus at the 17th China National Conference on Computational Linguistics, reaches 80. 28%. |
|
来源
|
电子学报
,2020,48(11):2162-2169 【核心库】
|
|
DOI
|
10.3969/j.issn.0372-2112.2020.11.010
|
|
关键词
|
自然语言处理
;
文本蕴含
;
自注意力机制
;
语义角色标注
;
预训练语言模型
|
|
地址
|
西北师范大学计算机科学与工程学院, 甘肃, 兰州, 730000
|
|
语种
|
中文 |
|
文献类型
|
研究性论文 |
|
ISSN
|
0372-2112 |
|
学科
|
自动化技术、计算机技术 |
|
基金
|
国家自然科学基金
;
甘肃省重点研发计划
|
|
文献收藏号
|
CSCD:6881572
|
参考文献 共
24
共2页
|
|
1.
郭茂盛. 文本蕴含关系识别与知识获取研究进展及展望.
计算机学报,2017,40(4):889-910
|
CSCD被引
15
次
|
|
|
|
|
2.
Ankur P P. A decomposable attention model for natural language inference.
Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing,2016:2249-2255
|
CSCD被引
1
次
|
|
|
|
|
3.
袁里驰. 利用配价信息的语义角色标注.
电子学报,2017,45(10):2533-2539
|
CSCD被引
3
次
|
|
|
|
|
4.
Shi C. Knowledge-based semantic embedding for machine translation.
Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers),2016:2245-2254
|
CSCD被引
1
次
|
|
|
|
|
5.
杨震. 基于语义的文本流形研究.
电子学报,2009,37(3):557-561
|
CSCD被引
8
次
|
|
|
|
|
6.
Peters M E. Deep contextualized word representations.
The 16th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,2018:2227-2237
|
CSCD被引
1
次
|
|
|
|
|
7.
Ranford A.
Improving language understanding by generative pre-training,2019
|
CSCD被引
1
次
|
|
|
|
|
8.
Devlin J. BERT: pre-training of deep bidirectional transformers for language understanding.
Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,2019:4171-4186
|
CSCD被引
227
次
|
|
|
|
|
9.
Cui Y. Pre-training with whole word masking for Chinese BERT.
arXiv: 1906. 08101,2019
|
CSCD被引
1
次
|
|
|
|
|
10.
Vaswani A. Attention is all you need.
Procceeding of 31st Conference on Neural Information Processing Systems (NIPS 2017),2017:6000-6010
|
CSCD被引
1
次
|
|
|
|
|
11.
Bowman S R. A large annotated corpus for learning natural language inference.
Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing,2015:632-642
|
CSCD被引
15
次
|
|
|
|
|
12.
翟延冬. 一种基于WordNet的短文本语义相似性算法.
电子学报,2012,40(3):617-620
|
CSCD被引
15
次
|
|
|
|
|
13.
袁里驰. 基于配价结构和语义依存关系的句法分析统计模型.
电子学报,2013,41(10):2029-2034
|
CSCD被引
4
次
|
|
|
|
|
14.
刘茂福. 基于事件语义特征的中文文本蕴含识别.
中文信息学报,2013,27(5):129-136
|
CSCD被引
11
次
|
|
|
|
|
15.
谭咏梅. 基于CNN与双向LSTM的中文文本蕴含识别方法.
中文信息学报,2018,32(7):11-19
|
CSCD被引
12
次
|
|
|
|
|
16.
Rocktaschel T. Reasoning about entailment with neural attention.
arXiv: 1509. 06664,2015
|
CSCD被引
1
次
|
|
|
|
|
17.
Wang S. Learning natural language inference with LSTM.
The 14th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,2016:1442-1451
|
CSCD被引
1
次
|
|
|
|
|
18.
Chen Q. Enhanced LSTM for natural language inference.
Proceedings of the 55th Annual Meeting of the Association for Computational,2017:1657-1668
|
CSCD被引
1
次
|
|
|
|
|
19.
刘广灿. 基于对抗正则化的自然语言推理.
自动化学报,2019,45(8):1455-1463
|
CSCD被引
2
次
|
|
|
|
|
20.
Pan B. Discourse marker augmented network with reinforcement learning for natural language inference.
Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers),2018:989-999
|
CSCD被引
2
次
|
|
|
|
|
了解气象卫星更多信息，请访问