Given a function , e.g.: , we can calculate its derivatives with respect to subsequent arguments (considering the rest of arguments as constants). Such derivatives are called partial derivatives and we denote them as etc. In this case: , , .
and the partial derivative exists if the limit exists.
Directional derivative is a generalization of partial derivatives. It defines the direction of the line tangent to the graph of the function in this direction. If the directional derivative at is
and it exists if the limit exists.
The derivative of a function at is a linear function , such that
exists and equals to zero.
If the derivative exists, then
The matrix of the derivative i.e.
is called the gradient of the function.
A function is differentiable at a given point if it has a derivative there. There is an important theorem, which states that if there partial derivatives exist inside of a ball around a given point and are continuous in this point, then the function is differentiable at this point.
We can consider functions , e.g: , . Then always this function can be considered as a sequence of funcrions , i.e.
In our example , and .
The only difference in the calculus is that here we have to calculate partial derivatives for each of functions . In particular, the derivative is a linear mapping , and its matrix
is called also the derivative or Jacobi’s matrix.
In our example, , we get
Derivative of composition
Similarly as in the one-dimensional case we get
Attention! this is multiplication of matrices (the order is important)!
The meaning of gradient
Let us consider a function again. Recall that , where . But . By the properties of a norm (Schwarz inequality) this value for any vector of unit length is biggest for . It means that is the direction of the biggest growth of the function at the given point. Moreover, the directional derivative shows the tangent of the angle of the tangent line in the given direction, so denotes the maximal growth of the function in the given point.