10

I know the Taylor series expansion in single variable case:

$$ f(x) = f(x_0) + f'(x_0)(x - x_0) + \frac{1}{2}f''(x_0)(x-x_0)^2 + \frac{1}{3!}f^{(3)}(x_0)(x-x_0)^3 + \frac{1}{4!}f^{(4)}(x_0)(x-x_0)^4 + \frac{1}{5!}f^{(5)}(x_0)(x-x_0)^5 + \dots $$

Multivariable case is stated as:

$$ f(\bar{x}) = f(\bar{x_0}) + \bar\nabla f(\bar{x_0})^T(\bar{x} - \bar{x_0}) + \frac{1}{2}(\bar{x} - \bar{x_0})^T\bar\nabla^2 f(\bar{x_0})(\bar{x} - \bar{x_0}) + H.O.T. $$

I can find the expression above everywhere, but it is not possible to find the open expression for the H.O.T. (at least I wasn't able to find it). No source states the explicit expression for the higher order terms. Do people avoid them for some reason? For example, are those terms too complex or too long to write?

So, what are these higher order terms in the multivariable case? Do they involve a $\bar\nabla^3 f(\bar{x_0})$, $\bar\nabla^4 f(\bar{x_0})$, $\bar\nabla^5 f(\bar{x_0})$, ... terms; if yes, how are they defined? Please write a few terms from H.O.T. to make the pattern clear.

hkBattousai
  • 4,543

4 Answers4

4

The terms are no different from those you'd expect based on the univariate case.

The general sum can be stated as

$$f(x) = \sum_{n=0}^\infty \frac{1}{n!}[(x-x_0) \cdot \nabla]^n f|_{x_0}$$

This follows from the single variable case by writing $x-x_0 = t \hat v$ for some unit vector $v$. Each component can then be expressed as a function of $t$ and expanded in a 1D Taylor series.

Muphrid
  • 19,902
3

The formula is almost identical to the single variable case. You should be able to find this in any good book covering calculus on Banach spaces.

Theorem. Let $E,F$ be Banach spaces, let $A\subseteq E$ be an open set and let $f:A\to F$ be a class $C^p$ map. Let $x\in A$ and let $v\in E$. Assume that the line segment $x+tv$ with $0\le t\le1$ is contained in $A$. Write $v^{(k)}$ for the $k$-tuple $(v,\dots,v)$. Then $$ f(x+v)=\sum_{k=0}^{p-1}\frac{f^{(k)}(x)v^{(k)}}{k!} + R_p, $$ where $$ R_p = \int_0^1 \frac{(1-t)^{p-1}}{(p-1)!} f^{(p)}(x+tv)v^{(p)}\,dt. $$

Proof. Integration by parts.

Here $f^{(k)}(x)\in L(E,L(E,\dots,L(E,F))\dots)$. (You should already know that the first derivative $f'(x)$ is just a linear map from $E$ to $F$, so we can differentiate the map $f':A\to L(E,F)$ to get $f'':A\to L(E,L(E,F))$ and so on.)

Check out Mathematical Analysis II by Zorich & Cooke or Real and Functional Analysis by Lang.

wj32
  • 4,236
2

They are just higher order tensors. Think of them multidimensional matrices. So (assuming the output of $f$ is just a real scalar), the first term in the Taylor expansion is just a scalar. The second term is a vector, all the first partials (AKA the gradient vector). The third term is a square matrix, all of the possible second partials (AKA the Hessian matrix). The fourth term is a 3D matrix, all of the possible third partials, and so on.

Fixed Point
  • 7,909
  • 3
  • 31
  • 48
2

In addition to below answers the explicit form for three variables are $$f(x,y,z)=f(x_0,y_0,z_0)$$ $$+\frac{\partial f_0}{\partial x}(x-x_0)+\frac{\partial f_0}{\partial y}(y-y_0)+\frac{\partial f_0}{\partial z}(z-z_0)\quad \Rightarrow Order 1$$ $$+\frac{1}{2} \bigg(\frac{\partial^2 f_0}{\partial x^2}(x-x_0)^2+\frac{\partial^2 f_0}{\partial y^2}(y-y_0)^2+\frac{\partial^2 f_0}{\partial z^2}(z-z_0)^2+2\frac{\partial^2 f_0}{\partial x\partial y}(x-x_0)(y-y_0) $$ $$+2\frac{\partial^2 f_0}{\partial x\partial z}(x-x_0)(z-z_0)+2\frac{\partial^2 f_0}{\partial z\partial y}(z-z_0)(y-y_0)\bigg)\quad \Rightarrow Order 2$$ And it goes like this to higher orders

AnilB
  • 4,202