Optimum Levenberg–Marquardt constant determination for nonlinear least-squares

A new method for determining an approximate optimum value for the Levenberg–Marquardt constant has been shown to improve the convergence rate of nonlinear least-squares problems including complex X-ray powder diffraction and single-crystal structural refinements. In the Gauss–Newton method of nonlinear least squares, a lower value for the objective function is occasionally not realized after solving the matrix equation AΔp = b. This situation occurs when either the objective function is at a minimum or the A matrix is ill conditioned. Invariably the Levenberg–Marquardt method is used, where the matrix equation is reformulated to (A + λI)Δp = b and λ is the Levenberg–Marquardt constant. The values chosen for λ depend on whether the objective function increases or decreases. This paper describes a new method for setting the Levenberg–Marquardt constant, as implemented in the computer program TOPAS-Academic Version 7, which in general results in an increased rate of convergence and additionally a lowering of the objective function as a function of starting parameter values. The reduction in computation is problem dependent and ranges from 10% for typical crystallographic refinements to 50% for large refinements. In addition, the method can be applied to general functions including cases where the objective function comprises both the sum of squares and penalties including functions with discontinuities. Of significance is the trivial extra computational effort required in determining λ as well as the simplicity in carrying out the calculation; the latter should allow for easy implementation in refinement programs.