帮助 关于我们

返回检索结果

Improve Robustness and Accuracy of Deep Neural Network with L_(2,∞) Normalization

查看参考文献25篇

Yu Lijia 1,2   Gao Xiaoshan 1,2 *  
文摘 In this paper,the L_(2,∞) normalization of the weight matrices is used to enhance the robustness and accuracy of the deep neural network (DNN) with Relu as activation functions.It is shown that the L_(2,∞) normalization leads to large dihedral angles between two adjacent faces of the DNN function graph and hence smoother DNN functions,which reduces over-fitting of the DNN.A global measure is proposed for the robustness of a classification DNN,which is the average radius of the maximal robust spheres with the training samples as centers.A lower bound for the robustness measure in terms of the L_(2,∞) norm is given.Finally,an upper bound for the Rademacher complexity of DNNs with L_(2,∞) normalization is given.An algorithm is given to train DNNs with the L_(2,∞) normalization and numerical experimental results are used to show that the L_(2,∞) normalization is effective in terms of improving the robustness and accuracy.
来源 Journal of Systems Science and Complexity ,2023,36(1):3-28 【核心库】
DOI 10.1007/s11424-022-1326-y
关键词 Deep neural network ; global robustness measure ; L_(2,∞) normalization ; over-fitting ; Rademacher complexity ; smooth DNN
地址

1. Academy of Mathematics and Systems Science,Chinese Academy of Sciences, Beijing, 100190  

2. University of Chinese Academy of Sciences, Beijing, 100049

语种 英文
文献类型 研究性论文
ISSN 1009-6124
学科 自动化技术、计算机技术
基金 supported by NKRDP ;  国家自然科学基金
文献收藏号 CSCD:7551062

参考文献 共 25 共2页

1.  LeCun Y. Deep learning. Nature,2015,521(7553):436-444 CSCD被引 3544 次    
2.  Voulodimos A. Deep learning for computer vision: A brief review. Comput. Intel. and Neurosc,2018 CSCD被引 1 次    
3.  Socher R. Deep Learning for NLP (without magic). Tutorial Abstracts of ACL'2012,2012:5 CSCD被引 1 次    
4.  Leshno M. Multilayer feedforward networks with a nonpolynomial activation function can approximate any function. Neural Networks,1993,6(6):861-867 CSCD被引 74 次    
5.  Goodfellow I. Deep Learning,2016 CSCD被引 710 次    
6.  Molchanov D. Variational dropout sparsifies deep neural networks. arXiv: 1701.05369,2017 CSCD被引 1 次    
7.  Srivastava N. Dropout: A simple way to prevent neural networks from overfitting. Journal of Machine Learning Research,2014,15:1929-1958 CSCD被引 1216 次    
8.  Wan L. Regularization of neural networks using DropConnect. ICML'13,2013,28:III-1058-III-1066 CSCD被引 1 次    
9.  Ioffe S. Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv: 1502.03167,2015 CSCD被引 68 次    
10.  Montufar G. On the number of linear regions of deep neural networks. NIPS'2014,2014 CSCD被引 1 次    
11.  Zhang X Y. Towards robust pattern recognition: A review. Proc. of the IEEE,2020,108(6):894-922 CSCD被引 9 次    
12.  Zheng S. Improving the robustness of deep neural networks via stability training. CVPR'16,2016:4480-4488 CSCD被引 1 次    
13.  Meng D. Improve robustness of sparse PCA by L1-norm maximization. Pattern Recognition,2012,45:487-497 CSCD被引 12 次    
14.  Hinton G. Distilling the knowledge in a neural network. arXiv: 1503.02531,2015 CSCD被引 80 次    
15.  Yu L. Robust and information-theoretically safe bias classifier against adversarial attacks. arXiv: 2111.04404,2021 CSCD被引 1 次    
16.  Yu L. Adversarial parameter attack on deep neural networks. arXiv: 2203.10502,2022 CSCD被引 1 次    
17.  Madry A. Towards deep learning models resistant to adversarial attacks. arXiv: 1706.06083,2017 CSCD被引 27 次    
18.  Lin W. Robustness verification of classification deep neural networks via linear programming. CVPR'2019,2019:11418-11427 CSCD被引 1 次    
19.  Carlini N. Towards evaluating the robustness of neural networks. IEEE Symposium on Security and Privacy CSCD被引 1 次    
20.  Neyshabur B. Norm-based capacity control in neural networks. COLT'15,2015:1376-1401 CSCD被引 1 次    
引证文献 1 篇

1 Chen Shaoshi Preface to the Special Topic on Computer Mathematics Journal of Systems Science and Complexity,2023,36(1):1-2
CSCD被引 0 次

显示所有1篇文献

论文科学数据集
PlumX Metrics
相关文献

 作者相关
 关键词相关
 参考文献相关

版权所有 ©2008 中国科学院文献情报中心 制作维护：中国科学院文献情报中心
地址：北京中关村北四环西路33号 邮政编码：100190 联系电话：(010)82627496 E-mail:cscd@mail.las.ac.cn 京ICP备05002861号-4 | 京公网安备11010802043238号