Training multilayer perceptrons Parameter by Parameter

Yan Lai Li; Kuan Quan Wang

Training multilayer perceptrons Parameter by Parameter

Yan Lai Li^*
, Kuan Quan Wang

^*Corresponding author for this work

Faculty of Computing

Harbin Institute of Technology

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

In this paper, a new fast training algorithm for multilayer perceptrons (MLP) is presented. This new algorithm, named Parameter by Parameter Optimization Algorithm (PBPOA), is proposed based on the idea of Layer By Layer (LBL) algorithm. The inputs errors of output layer and hidden layer are taken into consider. Four classes of solution equations for parameters of networks are deducted respectively. The presented algorithm doesn't need calculating the gradient of error function at all. In each iteration step, the weight or threshold can be optimized directly one by one with other variables fixed. Effectiveness of the presented algorithm is demonstrated by two benchmarks, in which faster convergence rate of training are obtained in contrast with the BP algorithm with momentum (BPM) and the conventional LBL algorithm.

Original language	English
Title of host publication	Proceedings of 2004 International Conference on Machine Learning and Cybernetics
Pages	3397-3401
Number of pages	5
State	Published - 2004
Event	Proceedings of 2004 International Conference on Machine Learning and Cybernetics - Shanghai, China Duration: 26 Aug 2004 → 29 Aug 2004

Publication series

Name	Proceedings of 2004 International Conference on Machine Learning and Cybernetics
Volume	6

Conference

Conference	Proceedings of 2004 International Conference on Machine Learning and Cybernetics
Country/Territory	China
City	Shanghai
Period	26/08/04 → 29/08/04

Keywords

Multilayer Perceptrons
Parameter By Parameter Optimization Algorithm (PBPOA)
Training algorithm

Cite this

@inproceedings{72f23d4e443448339fb8af49cca40289,

title = "Training multilayer perceptrons Parameter by Parameter",

abstract = "In this paper, a new fast training algorithm for multilayer perceptrons (MLP) is presented. This new algorithm, named Parameter by Parameter Optimization Algorithm (PBPOA), is proposed based on the idea of Layer By Layer (LBL) algorithm. The inputs errors of output layer and hidden layer are taken into consider. Four classes of solution equations for parameters of networks are deducted respectively. The presented algorithm doesn't need calculating the gradient of error function at all. In each iteration step, the weight or threshold can be optimized directly one by one with other variables fixed. Effectiveness of the presented algorithm is demonstrated by two benchmarks, in which faster convergence rate of training are obtained in contrast with the BP algorithm with momentum (BPM) and the conventional LBL algorithm.",

keywords = "Multilayer Perceptrons, Parameter By Parameter Optimization Algorithm (PBPOA), Training algorithm",

author = "Li, \{Yan Lai\} and Wang, \{Kuan Quan\}",

year = "2004",

language = "英语",

isbn = "0780384032",

series = "Proceedings of 2004 International Conference on Machine Learning and Cybernetics",

pages = "3397--3401",

booktitle = "Proceedings of 2004 International Conference on Machine Learning and Cybernetics",

note = "Proceedings of 2004 International Conference on Machine Learning and Cybernetics ; Conference date: 26-08-2004 Through 29-08-2004",

}

Li, YL & Wang, KQ 2004, Training multilayer perceptrons Parameter by Parameter. in Proceedings of 2004 International Conference on Machine Learning and Cybernetics. Proceedings of 2004 International Conference on Machine Learning and Cybernetics, vol. 6, pp. 3397-3401, Proceedings of 2004 International Conference on Machine Learning and Cybernetics, Shanghai, China, 26/08/04.

Training multilayer perceptrons Parameter by Parameter. / Li, Yan Lai; Wang, Kuan Quan.
Proceedings of 2004 International Conference on Machine Learning and Cybernetics. 2004. p. 3397-3401 (Proceedings of 2004 International Conference on Machine Learning and Cybernetics; Vol. 6).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Training multilayer perceptrons Parameter by Parameter

AU - Li, Yan Lai

AU - Wang, Kuan Quan

PY - 2004

Y1 - 2004

N2 - In this paper, a new fast training algorithm for multilayer perceptrons (MLP) is presented. This new algorithm, named Parameter by Parameter Optimization Algorithm (PBPOA), is proposed based on the idea of Layer By Layer (LBL) algorithm. The inputs errors of output layer and hidden layer are taken into consider. Four classes of solution equations for parameters of networks are deducted respectively. The presented algorithm doesn't need calculating the gradient of error function at all. In each iteration step, the weight or threshold can be optimized directly one by one with other variables fixed. Effectiveness of the presented algorithm is demonstrated by two benchmarks, in which faster convergence rate of training are obtained in contrast with the BP algorithm with momentum (BPM) and the conventional LBL algorithm.

AB - In this paper, a new fast training algorithm for multilayer perceptrons (MLP) is presented. This new algorithm, named Parameter by Parameter Optimization Algorithm (PBPOA), is proposed based on the idea of Layer By Layer (LBL) algorithm. The inputs errors of output layer and hidden layer are taken into consider. Four classes of solution equations for parameters of networks are deducted respectively. The presented algorithm doesn't need calculating the gradient of error function at all. In each iteration step, the weight or threshold can be optimized directly one by one with other variables fixed. Effectiveness of the presented algorithm is demonstrated by two benchmarks, in which faster convergence rate of training are obtained in contrast with the BP algorithm with momentum (BPM) and the conventional LBL algorithm.

KW - Multilayer Perceptrons

KW - Parameter By Parameter Optimization Algorithm (PBPOA)

KW - Training algorithm

UR - https://www.scopus.com/pages/publications/6344292223

M3 - 会议稿件

AN - SCOPUS:6344292223

SN - 0780384032

T3 - Proceedings of 2004 International Conference on Machine Learning and Cybernetics

SP - 3397

EP - 3401

BT - Proceedings of 2004 International Conference on Machine Learning and Cybernetics

T2 - Proceedings of 2004 International Conference on Machine Learning and Cybernetics

Y2 - 26 August 2004 through 29 August 2004

ER -

Training multilayer perceptrons Parameter by Parameter

Abstract

Publication series

Conference

Keywords

Other files and links

Fingerprint

Cite this