5. Derivatives of multivariable functions

Partial derivatives

Given a function f, e.g.: f(x,y,z)=2xy+z^2-y^2z+x^3, we can calculate its derivatives with respect to subsequent arguments (considering the rest of arguments as constants). Such derivatives are called partial derivatives and we denote them as \frac{\partial f}{\partial x} etc. In this case: \frac{\partial f}{\partial x}(x,y,z)=2y+3x^2, \frac{\partial f}{\partial y}(x,y,z)=2x-2yz, \frac{\partial f}{\partial z}(x,y,z)=2z-y^2.

So formally,

    \[\frac{\partial f}{\partial x_i}(a_1,\ldots, a_i,\ldots, a_n)=\lim_{h\to 0}\frac{f(a_1,\ldots, a_i+h,\ldots, a_n)-f(a_1,\ldots, a_i,\ldots, a_n)}{h}\]

and the partial derivative exists if the limit exists.

Directional derivative

Directional derivative is a generalization of partial derivatives. It defines the direction of the line tangent to the graph of the function in this direction. If v=(v_1,\ldots, v_n) the directional derivative at (a_1,\ldots, a_n) is

    \[\frac{\partial f}{\partial v}(a_1,\ldots, a_i,\ldots, a_n)=\]

    \[=\lim_{h\to 0}\frac{f((a_1,\ldots, a_i,\ldots, a_n)+h(v_1,\ldots, v_n))-f(a_1,\ldots, a_i,\ldots, a_n)}{h}\]

and it exists if the limit exists.

Derivative

The derivative of a function f at (a_1,\ldots, a_n) is a linear function Df_{(a_1,\ldots,a_n)}\colon \mathbb{R}^n\to \mathbb{R}, such that

    \[\lim_{(h_1,\ldots, h_n)\to(0,\ldots,0)}\]

    \[\frac{|f((a_1,\ldots, a_n)+(h_1,\ldots, h_n))-f(a_1,\ldots, a_n)-Df_{(a_1,\ldots, a_n)}(h_1,\ldots, h_n)|}{\|(h_1,\ldots, h_n)\|}\]

exists and equals to zero.

If the derivative exists, then

    \[Df_{(a_1,\ldots,a_n)}(h_1,\ldots, h_n)=\]

    \[=\frac{\partial f}{\partial x_1}(a_1,\ldots,a_n)\cdot h_1+\ldots +\frac{\partial f}{\partial x_n}(a_1,\ldots,a_n)\cdot h_n\]

The matrix of the derivative i.e.

    \[f'=\left[\frac{\partial f}{\partial x_1}(a_1,\ldots,a_n),\ldots, \frac{\partial f}{\partial x_n}(a_1,\ldots,a_n)\right],\]

is called the gradient of the function.

A function is differentiable at a given point if it has a derivative there. There is an important theorem, which states that if there partial derivatives exist inside of a ball around a given point and are continuous in this point, then the function is differentiable at this point.

Multi-dimensional values

We can consider functions f\colon \mathbb{R}^n\to \mathbb{R}^k, e.g: f(x,y)=(x+y,x-y), f\colon \R^2\to \R^2. Then always this function can be considered as a sequence of k funcrions f_i\colon \mathbb{R}^n\to \mathbb{R}, i.e.

    \[f=(f_1,\ldots, f_k)\]

In our example f_1(x,y)=x+y, and f_2(x,y)=x-y.

The only difference in the calculus is that here we have to calculate partial derivatives for each of functions f_i. In particular, the derivative is a linear mapping \mathbb{R}^n\to\mathbb{R}^k, and its matrix

    \[f'(a_1,\ldots, a_n)=\left[\begin{array}{ccc}\frac{\partial f_1}{\partial x_1}(a_1,\ldots,a_n)&\ldots& \frac{\partial f_1}{\partial x_n}(a_1,\ldots,a_n)\\\ldots & &\ldots \\\frac{\partial f_k}{\partial x_1}(a_1,\ldots,a_n)&\ldots& \frac{\partial f_k}{\partial x_n}(a_1,\ldots,a_n) \end{array}\right]\]

is called also the derivative or Jacobi’s matrix.

In our example, f(x,y)=(x+y,x-y), we get

    \[f'(x,y)=\left[\begin{array}{cc}1&1\\1&-1 \end{array}\right].\]

Derivative of composition

Similarly as in the one-dimensional case we get

    \[(g\circ f)'(a_1,\ldots, a_n)=g'(f(a_1,\ldots, a_n))\cdot f'(a_1,\ldots, a_n).\]

Attention! this is multiplication of matrices (the order is important)!

The meaning of gradient

Let us consider a function f\colon\matbb{R}^n\to\mathbb{\R} again. Recall that Df_{(a_1,\ldots,a_n)}(h_1,\ldots,h_n)=\frac{\partial f}{\partial v}(a_1,\ldots,a_n), where v=(h_1,\ldots,h_n). But Df_{(a_1,\ldots,a_n)}(h_1,\ldots,h_n)=\frac{\partial f}{\partial x_i}h_1+\ldots+\frac{\partial f}{\partial x_n}h_n=\langle f', v\rangle. By the properties of a norm (Schwarz inequality) this value for any vector v of unit length is biggest for v=f'/\|f\|. It means that f' is the direction of the biggest growth of the function at the given point. Moreover, the directional derivative shows the tangent of the angle of the tangent line in the given direction, so \frac{\partial f}{\partial (f'/\|f'\|)}=\langle f',f'/\|f'\|\rangle=\|f'\| denotes the maximal growth of the function in the given point.