基于深度学习的图像描述综述
Overview of Image Captions Based on Deep Learning
查看参考文献65篇
|
文摘
|
图像描述旨在通过提取图像的特征输入到语言生成模型中最后输出图像对应的描述,来解决人工智能中自然语言处理与计算机视觉的交叉领域问题——智能图像理解.现对2015—2020年间图像描述方向有代表性的论文进行汇总与分析,以不同核心技术作为分类标准将图像描述大致划分为基于Encoder-Decoder框架的图像描述、基于注意力机制的图像描述、基于强化学习的图像描述、基于生成对抗网络的图像描述和基于新融合数据集的图像描述五大类.使用NIC、Hard-Attention和Neural Talk三个模型在真实数据集MS-COCO数据集上进行实验,并从BLEU1、 BLEU2、BLEU3、BLEU4四处平均评分对比分析,展示三个模型效果.本文点明了未来图像描述的发展趋势,并指出了图像描述将要面临的挑战和可深入挖掘的研究方向. |
|
其他语种文摘
|
Image caption aims to extract the features of the image and input the description of the final output image into the language generation model, which solves the intersection of natural language processing and computer vision in artificial intelligence-image understanding. Summarize and analyze representative thesis of image description orientation from 2015 to 2020,different core technologies as classification criteria,it can be roughly divided into: image caption based on Encoder-Decoder framework, image caption based on attention mechanism, image caption based on reinforcement learning, image caption based on Generative Adversarial Networks, and based on new fusion data set these five categories. Use three models of NIC, Hard-Attention and Neural Talk to conduct experiments on the real data set MS-COCO data set, and compare the average scores of BLEU1, BLEU2, BLEU3, and BLEU4 to show the effects of the three models. This article points out the development trend of image caption in the future, and the challenges that image caption will face and the research directions that can be digged in. |
|
来源
|
电子学报
,2021,49(10):2048-2060 【核心库】
|
|
DOI
|
10.12263/DZXB.20200669
|
|
关键词
|
智能图像理解
;
Encoder-Decoder框架
;
注意力机制
;
强化学习
|
|
地址
|
1.
新疆大学软件工程技术重点实验室, 新疆, 乌鲁木齐, 830000
2.
新疆大学信息科学与工程学院, 新疆, 乌鲁木齐, 830000
|
|
语种
|
中文 |
|
文献类型
|
综述型 |
|
ISSN
|
0372-2112 |
|
学科
|
自动化技术、计算机技术 |
|
基金
|
国家自然科学基金
;
新疆维吾尔自治区自然科学基金
|
|
文献收藏号
|
CSCD:7090187
|
参考文献 共
65
共4页
|
|
1.
权宇. 融合深度扩张网络和轻量化网络的目标检测模型.
电子学报,2020,48(2):390-397
|
CSCD被引
20
次
|
|
|
|
|
2.
刘颖. 基于深度学习的小目标检测研究与应用综述.
电子学报,2020,48(3):590-601
|
CSCD被引
49
次
|
|
|
|
|
3.
杨K L.
Image caption的发展历程和最新工作的简要综述(2010-2018),2018
|
CSCD被引
1
次
|
|
|
|
|
4.
Vinyals O. Show and tell: Lessons learned from the 2015 MSCOCO image captioning challenge.
IEEE Transactions on Pattern Analysis and Machine Intelligence,2017,39(4):652-663
|
CSCD被引
35
次
|
|
|
|
|
5.
Tan X.
Multilingual neural machine translation with knowledge distillation,2019
|
CSCD被引
1
次
|
|
|
|
|
6.
.
亲历者.show_and_tell代码实现及测试-批量训练,2018
|
CSCD被引
1
次
|
|
|
|
|
7.
Karpathy A. Deep visual-semantic alignments for generating image descriptions.
IEEE Transactions on Pattern Analysis and Machine Intelligence,2017,39(4):664-676
|
CSCD被引
37
次
|
|
|
|
|
8.
Simonyan K.
Very deep convolutional networks for large-scale image recognition,2014
|
CSCD被引
554
次
|
|
|
|
|
9.
Fang H. From captions to visual concepts and back.
2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR),2015:1473-1482
|
CSCD被引
2
次
|
|
|
|
|
10.
Li N. Image cationing with visual-semantic LSTM.
Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence,2018:793-799
|
CSCD被引
1
次
|
|
|
|
|
11.
Anderson P. Bottom-up and topdown attention for image captioning and visual question answering.
2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition,2018:6077-6086
|
CSCD被引
12
次
|
|
|
|
|
12.
Fei Z C.
Better understanding hierarchical visual relationship for image caption,2019
|
CSCD被引
1
次
|
|
|
|
|
13.
Lee K H.
Learning visual relation priors for image-text matching and image captioning with neural scene graph generators,2019
|
CSCD被引
1
次
|
|
|
|
|
14.
Yao T. Hierarchy parsing for image captioning.
2019 IEEE/CVF International Conference on Computer Vision (ICCV),2019:2621-2629
|
CSCD被引
1
次
|
|
|
|
|
15.
He S. Image captioning through image transformer.
Computer Vision-ACCV 2020,2021:153-169
|
CSCD被引
1
次
|
|
|
|
|
16.
张红斌. 基于改进的有效区域基因选择与跨模态语义挖掘的图像属性标注.
电子学报,2020,48(4):790-799
|
CSCD被引
2
次
|
|
|
|
|
17.
Chen F H. GroupCap: group-based image captioning with structured relevance and diversity constraints.
2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition,2018:1345-1353
|
CSCD被引
1
次
|
|
|
|
|
18.
Pasunuru R. Multi-task video captioning with video and entailment generation.
Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers),2017:1273-1283
|
CSCD被引
1
次
|
|
|
|
|
19.
Zhou L W. Unified vision-language pre-training for image captioning and VQA.
Proceedings of the AAAI Conference on Artificial Intelligence,2020,34(7):13041-13049
|
CSCD被引
2
次
|
|
|
|
|
20.
Wang Y F. Skeleton key: Image captioning by skeleton-attribute decomposition.
2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR),2017:7378-7387
|
CSCD被引
1
次
|
|
|
|
|
了解气象卫星更多信息，请访问