A set (a curve in one-dimensional case, a surface — in two-dimensional case, etc.) will be called a manifold (officially: a submanifold embedded in ) of dimension (), if
where is a function of class such that the rows of matrix are linearly independent for any .
E.g. the sphere is a -dimensional manifold. Indeed,
where and . contains one row which is a non-zero vector for any point on the sphere.
By the implicit function theorem we immediately get that is locally at any poiny a graph of a function .
Given a function , and a manifold we may for example want to know what is the maximal value of this function on this manifold. More precisely, we are going to consider local extrema but with the function restricted only to the points on the manifold (so called conditional extrema). We shall say that is a conditional local maximum (respectively, minimum), if there exists a ball around , such that the largest value (respectively the least value) on the set of arguments the function takes at .
To find such points we are going to use the Lagrange multipliers Theorem. It states that given a function of class defined on an open set and a manifold defined as
where (niech ) jest funkcją klasy ,
if a point is a conditional local extremum, then there exist numbers , such that
Notice that this means that being a linear combination of is perpendicular to at , because the tangent space is the space perpendicular to all .
Thus given a compact manifold without a boundary we may use this condition to find all candidates for points on which the function can take its maximal and minimal value on the considered manifold.
For example let us take the manifold described by the equation and function . Then and . We get and . We look for , such that , i.e.
If then , thus , so , and . So we get two points in which we may have extrema: and (and ). The value at these points is . On the other hand, if , then and . Both cannot be equal to zero, so and , but since , we get . Thus, for we have points and , and for we have and . In these point the value of is respectively and . So the minimal value is and maximal is .
This is the necessary condition. But we can also formulate the sufficient condition for to be a conditional local extremum. Consider the mapping
Then if positive definite on the space tangent to at , then it is a conditional minimum. But if negative definite on the space tangent to at , then it is a conditional minimum.
If you want to find a maximum of a given function, eg.: in a given set, e.g. , you know how and can do it in two steps: find critical points in the interior of this set and then look for minima and maxima on the boundary of this set, which is a manifold, so we can use Lagrange multiplier here. The Kuhn-Tucker method does these two steps simultaneously.
Given function and a system of conditions , …, , function has a minimum in one of points which satisfy the following conditions:
- , where
- , …, ,
Notice that this is indeed it! In particular, if , then the condition about the derivatives of reduces to checking whether is zero, i.e. finding critical points of . But if , we also include equation and we add the derivatives of multiplied by to the equations about derivatives — so we are doing exactly the same which we would do using Lagrange multipliers.
In our example we are looking for maximum inside . Which is the same as finding the minimum of , and we have
, and . Thus,
and we are looking for points such that
- , , ,
Thus, if , the value of is which also equals to the value of . If , then the equation implies that , which is a contradiction. If , we get , so the value of is , and the value of is . For , similarly we get a contradiction. So we get that the maximum of is .