I'm taking a machine learning course, which involves taking a lot of analytical gradients and Hessians. It would be ideal if I could perform these calculations in Mathematica. However, I am only aware of how to do this for scalar functions, not ones that involve matrices. Could anybody assist me with this or point me in the right direction?
For example, say I have
$$g(\boldsymbol{w})=-\cos(2\pi\boldsymbol{w}^{T}\boldsymbol{w})+\boldsymbol{w}^{T}\boldsymbol{w}$$
I want to have Mathematica be able to compute, symbolically, that the gradient is
$$\nabla g(\boldsymbol{w})=\sin(2\pi\boldsymbol{w}^{T}\boldsymbol{w})4\pi\boldsymbol{w}+2\boldsymbol{w}$$
and, ideally, that the Hessian is
$$\nabla^2g(\boldsymbol{w})=(4\pi)^{2}\cos(2\pi\boldsymbol{w}^{T}\boldsymbol{w})\boldsymbol{w}\boldsymbol{w}^{T}+(2+4\pi\sin(2\pi\boldsymbol{w}^{T}\boldsymbol{w}))\boldsymbol{I}$$
D, you can use vector and tensor valued gradients. For exampled = 3; w = Array[wc, {d}]; g = w.w; D[g, {w, 2}]– Mauricio Fernández Apr 16 '17 at 09:43