Positive and negative definite forms over
Given a bilinear real form , we can classify it with respect to possible sign of results on a single vector:
- form is positive definite, if for all non-zero , we get .
- form is negative definite, if for all non-zero , we get .
- form is positive semidefinite, if for all , we get .
- form is negative semidefinite, if for all , we get .
Obviously a form may not fall in any of those categories, if for some we have and . Such forms are called indefinite.
Notice that a form is positive definite if and only if , and is negative definite if and only if , thus if the numbers on the diagonal in a diagonal congruent matrix are strictly positive (positive definite) or strictly negative (negative definite) respectively. In order to find out whether a form is positive or negative definite or semidefinite we may find an orthogonal basis and the matrix of the form in this basis.
Alternatively, we may complement a formula of a form to squares making sure to use all expressions with the first variable first, and then all with the second one, and so on.
where , i , so the form is non-definite. The basis , in which the formula is expressed is , because
Since the sign of the determinant does not change in congruent matrix, we obtain so called Sylvester’s criterion which determines whether a form is positively definite or negatively definite. Notice that it does not tell anything about the categories with semidefinite forms!
How does it work? We study determinants of minors: let be the matrix of size in the left upper corner of the matrix of a form we study. Let be the size of the matrix of this form. Sylvester’s criterion consists of the two following facts:
- if for any we have , the form is positively definite,
- if for any we have for even , and , for odd , then the form is negatively definite.
E.g. let be such that its matrix is , so and , therefore and , so form is positively definite.
E.g., let now be such that its matrix is , so and , also , therefore and and , so form is negatively definite.
Finally let be such that its matrix is , so and , therefore and , so form is neither positively definite nor negatively definite.
A symmetric positive definite bilinear form over is called a scalar product. Scalar products enable us to define angles between vectors and lengths of vectors.
Summing up the definitions a scalar product is a function mapping vectors to , such that:
- for every non-zero , .
Standard scalar product in is sum of products on subsequent places, so e.g.: .
A linear space over with a scalar product is called an euclidean space.
Length of a vector and angles between vectors
By Pitagoras Theorem it is easy to see (for the standard product) that is the square of the length of a vector, e.g. . The length of a vector , also called the norm of , will be denoted as . This motivates the definition:
Assume now that we are given three vectors and forming a triangle. So . Let be the angle between i . The law of cosines states that:
So cosine of an angle between vectors is defined by the following formula:
One more application of scalar product is calculating perpendicular projection of a vector onto a direction given by a second vector. Let be the perpendicular projection of onto direction given by . It will have the same direction as and length , where is the angle between and . Therefore:
because: is the vector of length in direction of .
Properties of norm and product
It is not very hard to prove the following
- if and only if is the zero vector
- (Schwarz inequality)
- (triangle inequality)
- jeśli , to (Pitagoras Theorem)
Perpendicularity, perpendicular spaces
We know that two vectors are perpendicular if cosine of the angle between them equals zero. Therefore if and only if .
Notice that if we would like to find all vectors perpendicular to , then the above is the equation we have to solve. Moreover this is a linear uniform equation. If we would like to find the set of vectors perpendicular to all vectors from a given list, then we will get a system of uniform linear equations. So given a linear subspace , the set (called the orthogonal complement of ) of all vectors perpendicular to all the vectors from is also a linear subspace! It is the space of solutions of some system of linear equations.
For example, let in the space with standard product. A vector is perpendicular to those vectors (and so also to every vector of ), if and , in other words, if it satisfies the following system of equations:
So the general solution has the following form: , and therefore we have the following basis of : .
Notice that the coefficients in a system of equations describing given linear space are vectors which span the perpendicular space! Which is a new insight into our method of finding a system of equations for a space given by its spanning vectors.
Orthogonal and orthonormal bases
A basis of a space will be called orthogonal, if every pair of vectors in it are perpendicular. E.g. is an orthogonal basis of with standard product — indeed it is easy to check, that all pairs are perpendicular, e.g. .
We will learn how to find such basis later on. Now let us notice that calculating coordinates of a vector in such a basis is fairly simple. Actually, we are calculating its projections onto vectors from the basis. Given vector , its -th coordinate in an orthogonal basis, which has as its -th vector is simply . Therefore, coordinates of in the above exemplary basis are: , and .
Orthonormal basis is an orthogonal basis in which additionally all the vectors have length . Given an orthogonal basis we can simply divide each of the vectors by its length to get an orthonormal basis. So is an orthonormal basis of . Notice that calculating the coordinates of a vector in an orthonormal basis is even simpler. Since , we get that the -th coordinate of is simply , where is the -th vector of the basis. Therefore the coordinates of in the basis in the example are: , and .
Projection onto a linear subspace
We already know how to calculate a projection of a vector onto a line defined by a given vector. To calculate a projection onto more-dimensional subspace we have to calculate its orthogonal basis and calculate projections onto directions defined by each of the vectors from the basis and sum those projections.
Notice also that if is a projection of a vector onto , and is its projection onto , then . Which can be nicely used when calculating a projection of a vector onto a plane in the case of three-dimensional space. Indeed, if is a plane in , then the perpendicular space is a line and is given by a vector. Denote this vector by . Therefore the projection of onto plane is:
Orthogonal reflection across a linear subspace
We can use our ability to calculate the projection of onto , to easily calculate (see the image below) its image under reflection across :
Assume that we are given a space spanned by some vectors , e.g. . We would like to find an orthogonal basis of this space. The method of finding such a basis is called Gram-Schmidt orthogonalization. The idea is to take as the first vector of the new basis the first vector from the original basis:
The second vector has to be be perpendicular to the first one, so we take the second vector and subtract its projection onto the first one. So:
so only perpendicular ,,part” is left. In the case of the third vector we need to subtract projections onto both already constructed vectors:
In the case of more dimensions we will continue this procedure further.
In our case in the space with standard product:
So the basis we were looking for is (I can drop the fractions since multiplication by a number does not change angles).
Linear maps: projection onto a linear subspace and reflection across a linear subspace
Notice that a projection onto a linear subspace and reflection across are linear mappings. Moreover, it is easy to see its eigenvectors:
- since projection does not change vectors in , they are eigenvectors with eigenvalue . On the other hand, vectors from are multiplied by zero, so they are eigenvectors with eigenvalue zero.
- since reflection does not change vectors in , they are eigenvectors with eigenvalue . On the other hand, vectors from are multiplied by , so they are eigenvectors with eigenvalue .
So basis consisting of vectors from a basis of and of vectors from a basis of is a basis of eigenvectors of both those maps. Which make it possible to calculate their formulas.
E.g. let in the space with standard product. Therefore basis is . So if is the projection onto , and is the reflection across , then are eigenvectors with eigenvalue of both maps. Also is an eigenvector with eigenvalue zero for , and for . Therefor basis is a basis of eigenvectors of both maps, and::
Let us calculate their formulas. We have:
Given a system of vectors in an euclidean space, the matrix is called the Gram’s matrix of this system of vectors and its determinant is called Gram’s determinant . Immediately on can notice, that Gram’s matrix is symmetrical.
Notice in particular, that if the columns of a matrix contain the coordinates of those vectors in a othonormal basis, then . Thus, if the number of vectors equals the dimension of the whole space (i.e. if is a square matrix), then .
In particular, always . And it is equal to zero if and only if the system of vectors is linearly dependent.