The Davidon–Fletcher–Powell formula (or DFP; named after William C. Davidon, Roger Fletcher, and Michael J. D. Powell) finds the solution to the secant equation that is closest to the current estimate and satisfies the curvature condition. It was the first quasi-Newton method to generalize the secant method to a multidimensional problem. This update maintains the symmetry and positive definiteness of the Hessian matrix.
Given a function f(x), its gradient ( \( \nabla f \) ), and positive-definite Hessian matrix B {\displaystyle B} B, the Taylor series is
\( {\displaystyle f(x_{k}+s_{k})=f(x_{k})+\nabla f(x_{k})^{T}s_{k}+{\frac {1}{2}}s_{k}^{T}{B}s_{k}+\dots ,} \)
and the Taylor series of the gradient itself (secant equation)
\( {\displaystyle \nabla f(x_{k}+s_{k})=\nabla f(x_{k})+Bs_{k}+\dots } \)
is used to update B.
The DFP formula finds a solution that is symmetric, positive-definite and closest to the current approximate value of \( B_{k} \) :
\( {\displaystyle B_{k+1}=(I-\gamma _{k}y_{k}s_{k}^{T})B_{k}(I-\gamma _{k}s_{k}y_{k}^{T})+\gamma _{k}y_{k}y_{k}^{T},} \)
where
\( {\displaystyle y_{k}=\nabla f(x_{k}+s_{k})-\nabla f(x_{k}),} \)
\( {\displaystyle \gamma _{k}={\frac {1}{y_{k}^{T}s_{k}}},} \)
and \( B_{k} \) is a symmetric and positive-definite matrix.
The corresponding update to the inverse Hessian approximation \( {\displaystyle H_{k}=B_{k}^{-1}} \) is given by
\( {\displaystyle H_{k+1}=H_{k}-{\frac {H_{k}y_{k}y_{k}^{T}H_{k}}{y_{k}^{T}H_{k}y_{k}}}+{\frac {s_{k}s_{k}^{T}}{y_{k}^{T}s_{k}}}.} \)
B is assumed to be positive-definite, and the vectors\( s_{k}^{T} \) and y must satisfy the curvature condition
\( {\displaystyle s_{k}^{T}y_{k}=s_{k}^{T}Bs_{k}>0.} \)
The DFP formula is quite effective, but it was soon superseded by the Broyden–Fletcher–Goldfarb–Shanno formula, which is its dual (interchanging the roles of y and s).[1]
See also
Newton's method
Newton's method in optimization
Quasi-Newton method
Broyden–Fletcher–Goldfarb–Shanno (BFGS) method
Limited-memory BFGS method
Symmetric rank-one formula
Nelder–Mead method
References
Avriel, Mordecai (1976). Nonlinear Programming: Analysis and Methods. Prentice-Hall. pp. 352–353. ISBN 0-13-623603-0.
Further reading
Davidon, W. C. (1959). "Variable Metric Method for Minimization". AEC Research and Development Report ANL-5990. doi:10.2172/4252678.
Fletcher, Roger (1987). Practical methods of optimization (2nd ed.). New York: John Wiley & Sons. ISBN 978-0-471-91547-8.
Kowalik, J.; Osborne, M. R. (1968). Methods for Unconstrained Optimization Problems. New York: Elsevier. pp. 45–48. ISBN 0-444-00041-0.
Nocedal, Jorge; Wright, Stephen J. (1999). Numerical Optimization. Springer-Verlag. ISBN 0-387-98793-2.
Walsh, G. R. (1975). Methods of Optimization. London: John Wiley & Sons. pp. 110–120. ISBN 0-471-91922-5.
Undergraduate Texts in Mathematics
Graduate Studies in Mathematics
Hellenica World - Scientific Library
Retrieved from "http://en.wikipedia.org/"
All text is available under the terms of the GNU Free Documentation License
