Levenberg-Marquardt Method

Levenberg-Marquardt is a popular alternative to the Gauss-Newton method of finding the minimum of a function that is a sum of squares of nonlinear functions,

Let the Jacobian of be denoted , then the Levenberg-Marquardt method searches in the direction given by the solution to the equations

where are nonnegative scalars and is the identity matrix. The method has the nice property that, for some scalar related to , the vector is the solution of the constrained subproblem of minimizing subject to (Gill et al. 1981, p. 136).
The method is used by the command FindMinimum[f, x, x0] when given the Method -> LevenbergMarquardt option.

SEE ALSO: Minimum, Optimization

REFERENCES:

Bates, D. M. and Watts, D. G. Nonlinear Regression and Its Applications. New York: Wiley, 1988.

Gill, P. R.; Murray, W.; and Wright, M. H. "The Levenberg-Marquardt Method." §4.7.3 in Practical Optimization. London: Academic Press, pp. 136-137, 1981.

Levenberg, K. "A Method for the Solution of Certain Problems in Least Squares."Quart. Appl. Math. 2, 164-168, 1944.

Marquardt, D. "An Algorithm for Least-Squares Estimation of Nonlinear Parameters."SIAM J. Appl. Math. 11, 431-441, 1963.

Referenced on Wolfram|Alpha: Levenberg-Marquardt Method

CITE THIS AS:

Weisstein, Eric W. "Levenberg-Marquardt Method." From MathWorld--A Wolfram Web Resource. http://mathworld.wolfram.com/Levenberg-MarquardtMethod.html