基于动量方法的受限玻尔兹曼机的一种有效算法
An Effective Algorithm of Restricted Boltzmann Machine Based on Momentum Method
查看参考文献25篇
|
文摘
|
深度学习给模式识别与机器学习带来了巨大的变化,已成功应用于语言处理、图像处理、信号处理、商业经济等方面.受限玻尔兹曼机(Restricted Boltzmann Machine,RBM)是一个表示能力强、很好的生成模型,多个RBM堆叠而构成的深度信念网络模型(Deep Belief Nets,DBN)的学习时间会较长.为加快整个DBN网络的学习时间和提高分类效果,本文提出基于动量方法RBM的一种有效算法.该算法在RBM预训练阶段,结合梯度上升算法特点采取快速上升的动量方式;以及BP算法微调阶段,为了能精确的找到最优点,结合梯度下降算法特点,相应的引入缓慢下降式的动量项,即在梯度上升和梯度下降过程中都使用不同的动量方式.本文算法在MNIST手写数字体和CMU-PIE人脸数据库上进行了实验,结果表明,提出的改进算法能够有效地增强图像特征的表达能力,提高图像的分类效果和实验效率. |
|
其他语种文摘
|
Deep learning is bringing revolution to pattern recognition and machine learning,which has been successfully applied to language processing, image processing, signal processing,business economy and so on. Restricted Boltzmann machine (RBM) is a strong representation and generative mod el,however, the learning time of deep belief nets (DBN), which consists of multiple stacking RBM,will be longer. In this paper, the improved momentum method is used not only in gradient ascent algorithm but also in gradient descent algorithm for both classification accuracy enhancement and training time decreasing. According to the characteristics of the gradient ascent algorithm,a rapidly ascending momentum method is used in the RBM pre-training phase,which greatly improves the speed of learning. According to the characteristics of the gradient descent algorithm, an improved slowly descending momentum term is also used in the fine-tuning stage to accurately find the best point. Through the recognition experiments on the MNIST dataset and CMU-PIE face dataset, the achieved results show that the improved momentum algorithm can effectively enhance the ability of image feature expression and improve both accuracy and computation efficiency. |
|
来源
|
电子学报
,2019,47(1):176-182 【核心库】
|
|
DOI
|
10.3969/j.issn.0372-2112.2019.01.023
|
|
关键词
|
深度学习
;
受限玻尔兹曼机
;
Kullback- Leibler(KL)距离
;
蒙特卡罗思想
;
动量
|
|
地址
|
1.
(武汉)中国地质大学数理学院, 湖北, 武汉, 430074
2.
湖北经济学院信息管理与统计学院, 湖北, 武汉, 430205
3.
中国地质大学(武汉), 地球内部多尺度成像湖北省重点实验室, 湖北, 武汉, 430074
|
|
语种
|
中文 |
|
文献类型
|
研究性论文 |
|
ISSN
|
0372-2112 |
|
学科
|
自动化技术、计算机技术 |
|
基金
|
湖北省教育厅科技处重点项目
;
湖北省高等学校优秀中青年创新团队计划项目
|
|
文献收藏号
|
CSCD:6437287
|
参考文献 共
25
共2页
|
|
1.
焦李成. 神经网络七十年:回顾与展望.
计算机学报,2016,39(1):1-21
|
CSCD被引
4
次
|
|
|
|
|
2.
Lee H. Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations.
Proceedings of the 26th Annual International Conference on Machine Learning,2009:609-616
|
CSCD被引
28
次
|
|
|
|
|
3.
Swersky K. A tutorial on stochastic approximation algorithms for training restricted boltzmann machines and deep belief nets.
ITA,2010:80-89
|
CSCD被引
1
次
|
|
|
|
|
4.
Mei X G. Infrared ultraspectral signature classification based on a restricted Boltzmann machine with sparse and prior constraints.
International Journal of Remote Sensing,2015,36(18):4724-4747
|
CSCD被引
2
次
|
|
|
|
|
5.
Hinton G E.
Improving neural networks by preventing coadaptation of feature detectors,2012
|
CSCD被引
5
次
|
|
|
|
|
6.
Wager S.
Dropout training as adaptive regularization,2013
|
CSCD被引
1
次
|
|
|
|
|
7.
Hinton G E. Training products of experts by minimizing contrastive divergence.
Neural Computation,2002,14(8):1711-1800
|
CSCD被引
276
次
|
|
|
|
|
8.
Mayraz G. Recognizing handwritten digits using hierarchical products of experts.
IEEE Transactions on Pattern Analysis and Machine Intelligence,2002,24(2):189-197
|
CSCD被引
4
次
|
|
|
|
|
9.
Hinton G E. A fast learning algorithm for deep belief nets.
Neural Computation,2006,18(7):1527-1554
|
CSCD被引
1398
次
|
|
|
|
|
10.
杨杰. 基于弱监督学习的去噪受限玻尔兹曼机特征提取算法.
电子学报,2014,42(12):2365-2370
|
CSCD被引
3
次
|
|
|
|
|
11.
Lopes N. Restricted Boltzmann machines and deep belief networks on multi-core processors.
WCCI 2012 IEEE World Congress on Computational Intelligence June,2012:10-15
|
CSCD被引
2
次
|
|
|
|
|
12.
Zhang Ch Y. MapReduce based distributed learning algorithm for restricted Boltzmann machine.
Neurocomputing,2016,198:4-11
|
CSCD被引
2
次
|
|
|
|
|
13.
Rumelhart D E. Learning representations by back-propagating errors.
Nature,1986,323:533-536
|
CSCD被引
1028
次
|
|
|
|
|
14.
Hinton G E.
A practical guide to training restricted Boltzmann machines. Neural Networks: Tricks of the Trade (2nd ed),2012:599-619
|
CSCD被引
1
次
|
|
|
|
|
15.
Sutskever I. On the importance of initialization and momentum in deep learning.
Proc International Conference on Machine Learning,2013:1139-1147
|
CSCD被引
1
次
|
|
|
|
|
16.
Nitanda A. Stochastic proximal gradient descent with acceleration techniques.
Proc Advances in Neural Information Processing Systems,2014:1574-1582
|
CSCD被引
1
次
|
|
|
|
|
17.
Zareba S. Accelerated learning for restricted Boltzmann machine with momentum term.
International Conference on Systems Engineering,2015:187-192
|
CSCD被引
2
次
|
|
|
|
|
18.
Yuan K. On the influence of momentum acceleration on online learning.
Journal of Machine Learning Research,2016(17):1-66
|
CSCD被引
3
次
|
|
|
|
|
19.
李飞. 基于权值动量的加速学习算法研究.
自动化学报,2017,43(7):1142-1159
|
CSCD被引
7
次
|
|
|
|
|
20.
Fischer A. Training restricted Boltzmann machines: An introduction.
Pattern Recognition,2014(47):25-39
|
CSCD被引
39
次
|
|
|
|
|
了解气象卫星更多信息，请访问