Is there any automatic differentiation package?

Question

I'm wondering if an automatic differentiation package exists for Mathematica. This is what I mean by automatic differentiation.

Answer 1

For anyone still interested in this: I'm working on an implementation of dual numbers for Mathematica which should allow you to calculate automatic derivatives of (hopefully) many programs.

Check it out on GitHub.

Here's a quick example of how to obtain the derivative of a programmatic function with a While loop (adapted from the documentation). It's a fixed point algorithm to solve an equation:

f[a_?NumericQ, x0 : _?NumericQ : 1.0] := 
 Module[{ x = x0, y, i = 0},
  While[(y = Cos[a x]) != x,
   x = y;
   i++
   ];
  x
 ];

It doesn't have a symbolic derivative:

Derivative[1] @ f
(* Derivative[1][f] *)

You can find the derivative of its first argument simply by passing a Dual with non-standard part equal to 1:

<<DualNumbers`
f[1.]
f[Dual[1., 1.]]
(* 0.739085 *)
(* Dual[0.739085, -0.297474] *)

The first argument of Dual is the function value and the second argument is the derivative at that point. Let's convince ourselves that this derivative is correct:

With[{h = 0.001, a = 1.0},
 (f[a + h] - f[a - h])/(2 h)
 ]
(* -0.297474 *)

Let's also check the derivative of the second argument:

f[1., Dual[1., 1.]]
(* Dual[0.739085, 1.86543*10^-14] *)

The derivative of the second argument is pretty much zero. This makes sense, of course, since small variations in the initial value in the fixed point search shouldn't influence the result.

Answer 2

Try this. It may not be exactly what your looking for, but it also may give you a good starting point.