In mathematics and computer algebra, automatic differentiation is a method to evaluate the derivative of a function, at a given point. Two classical ways of doing this are:

• symbolically differentiate the function and evaluate it in the point; or
• use finite differences.

The drawback with symbolic differentiation is low speed. Many functions grow too complex as higher derivatives are calculated. Two important drawbacks with finite differences are round-off errors in the discretization process, and cancellation. Thus both classical methods have problems with calculating higher derivatives, where the complexity and errors increase. Automatic differentiation solves all of the mentioned problems and is easily implemented on computers.

First-order automatic differentiation

A new arithmetic is introduced using ordered pairs as objects. The new arithmetic uses ordinary arithmetic on the first element of the pair and a first order differentiation arithmetic applying the chain rule on the second element. The basic arithmetic operations and some standard functions are defined by

The derivative of a function at the point x₀ is now calculated by substituting x₀ by the ordered pair in the expression for . A constant c is substituted by . Calculating using the above arithmetic gives as the answer.

Higher order automatic differentiation

The above arithmetic can be generalized, in the natural way, to second order and higher derivatives. However, the arithmetic rules quickly grow very complicated. Instead, truncated Taylor series arithmetic is used. This is possible because the Taylor summands in a Taylor series of a function are products of known coefficients and derivatives of the function.

Software

There are plenty of libraries for automatic differentiation.

See also

• Dual numbers

External link

An "entry site to everything you want to know about automatic differentiation"