自顶向下优化神经网络的方法

广东工业大学学报 ›› 2006, Vol. 23 ›› Issue (3): 95-101.

自顶向下优化神经网络的方法

广东商学院信息学院广东广州510320；

出版日期:2006-06-02 发布日期:2006-06-02

A Heuristic Approach for Improving Performance of Neural Network with the Optimal Brain Surgeon

(School of Information Science and Technology,Guangdong University of Business Studies,Guangzhou 510320,China)

Online:2006-06-02 Published:2006-06-02

摘要/Abstract

摘要： 介绍了一种自动优化神经网络的新启迪方法.这种启迪方法综合采用了快速自顶向下优化神经网络结构算法、动态优化学习参数算法和快速交叉校验算法.首先,快速自顶向下优化神经网络结构算法自动地优化神经网络结构;其次,动态优化学习参数算法动态地调整学习参数和选取优化的学习参数;随后,快速交叉校验算法的引入能有效地防止过度适应问题.实验结果显示,这种启迪方法与其它算法相比,具有更强的归纳性能、优化的网络结构和更快的学习速度.

关键词: 神经网络；神经中枢手术优化；自顶向下；学习参数优化；交叉校验；归纳；

Abstract: A new heuristic approach is proposed for the automated optimization of neural network.It combines fast optimal brain surgeon,dynamic optimization of learning parameters and fast cross-validation.The fast optimal brain surgeon optimizes architecture of neural network automatically.The dynamic optimization of learning parameters is able to dynamically vary its learning parameters and select optimal learning parameters.The fast cross-validation provides improvements of avoiding overfitting problem effectively.Experimental results demonstrate that with the heuristic approach,considerable performance gains are obtained compared to the other algorithms.This includes better generalization,optimal network architecture,and faster learning.

Key words: neural network； optimal brain surgeon； top-down； learning parameters optimization； cross-validation； generalization；

杨钟瑾； . 自顶向下优化神经网络的方法[J]. 广东工业大学学报, 2006, 23(3): 95-101.

YANG Zhong-jin . A Heuristic Approach for Improving Performance of Neural Network with the Optimal Brain Surgeon[J]. Journal of Guangdong University of Technology, 2006, 23(3): 95-101.

参考文献

[1] 杨钟瑾,史忠科. 快速选定神经网络优化结构的方法[J]. 计算机科学. 2005(04)

[2] 杨钟瑾,史忠科. 快速自顶向下优化神经网络结构的方法[J]. 系统仿真学报. 2005(01)

[1] (美)MartinT.Hagan等著.神经网络设计[M]. 机械工业出版社, 2002

[2] 史忠科著.神经网络控制理论[M]. 西北工业大学出版社, 1997

[1] Kung S,Fallside F,Sorenson J A,et at.Neural networks for signal processing. Proceed ing of the 1992 IEEEW orkshop . 1992

[2] Rumelhart DE,Hinton GE,Williams RJ.Learning internal representations by error propagation. Parallel Distributed Processing: Exploration in the Microstructures of Cognition . 1986

[3] Simon Haykin.Neural Networks. . 1994

[4] Fahlaman SE,Lebiere C.The cascade-correlation learning architecture. Advances in Neural Information Processing Systems . 1990

[5] Hassibi B,Stork D G,Wol G J.Optimal brain surgeon and general network pruning. IEEE International Conference on Neural Networks . 1993

[6] Holland John H.Adaptation in natural and artificial system. . 1975

[7] Yu X H,Chen G A,Cheng S X.Dynamic learning rate optimization of the back propagation algorithm. IEEE Transactions on Neural Networks . 1995

[8] Blum E K,Li L K.Approximation theory and feedforward networks. Neural Networks . 1991

[9] Fletcher R.Practical Methods of Optimization. . 1990

[10] Guyon I,Wang P S P.Special Issue on Neural Networks and Pattern Recognition. Pattern Recognition Artificial Intelligence . 1993

[11] FRIEDMAN J H.An overview of predictive learning and function approximation. From statisticsto neural networks:theory and pattern recognition applications,Proceeding of the ASI Conference,Subseries F . 1994

[12] MOODY J.Prediction risk and architecture selection for neural networks. From Sta-tistics to Neural Networks:Theory and Pattern Recognition Applications,NATO ASI Series F . 1994

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed