Performance Of Scaled Conjugate Gradient Algorithm In Face Recognition

Mohd Afzanizam , Hamdan (2009) Performance Of Scaled Conjugate Gradient Algorithm In Face Recognition. Project Report. UTeM, Melaka,Malaysia. (Submitted)

	PDF (24 Pages) Performance_Of_Scaled_Conjugate_Gradient_Algorithm_In_Face_Recognition_Mohd_Afzanizam_Bin_Hamdan_TA1650.M33_2009_-_24_Pages.pdf - Submitted Version Download (341kB)
	PDF (Full Text) Performance_Of_Scaled_Conjugate_Gradient_Algorithm_In_Face_Recognition_Mohd_Afzanizam_Bin_Hamdan_TA1650.M33_2009.pdf - Submitted Version Restricted to Registered users only Download (1MB)

Abstract

A supervised learning algorithm (Scaled Conjugate Gradient, SCG) with superliner convergence rate is introduced. The algorithm is based upon a class of optimization techniques well known in numerical analysis as the Conjugate Gradient Methods. SCG uses second order information from the neural network but requires only (N) memory usage, where N is the number of weights in the network. The performance of SCG yields a speed-up of at least an order of magnitude relative to BP. The speed-up depends on the convergence criterion, i.e., the bigger demand for reduction in error the bigger the speed-up. SCG is fully automated including no user dependent parameters and avoids a time consuming line-search, which CGB and BFGS uses in each iteration in order to determine an appropriate step size. The smaller the complexity of the neural network relative to the problem domain, the bigger the possibility that the weight space contains long ravines characterized by sharp curvature. While BP is inefficient on these ravine phenomena, it is shown that SCG handles them effectively.

Actions (login required)

View Item