5. Euclidean spaces

Part 1.: problems, solutions.
Part 2.: problems, solutions.
Part 3.: problems, solutions.

Scalar product

To study angles between vectors it will be convenient to use scalar product. A scalar (inner) product of two vectors \alpha,\beta, is a function \left<\alpha,\beta\right> such that:

  • \langle au+bv,w\rangle=a\langle u,w\rangle +b\langle v,w\rangle,
  • \langle u,v\rangle=\langle v,u\rangle,
  • for every non-zero u, \langle u,u\rangle>0.

Standard scalar product in \mathbb{R}^n is sum of products on subsequent places, so e.g.: \left<(1,2,-1),(2,0,1)\right>=1\cdot 2+2\cdot 0+(-1)\cdot 1=2+0-1=1.

Euclidean space

A linear space over \mathbb{R} with a scalar product is called an euclidean space.

Length of a vector and angles between vectors

By Pitagoras Theorem it is easy to see (for the standard product) that \left<\alpha,\alpha\right> is the square of the length of a vector, e.g. \left<(3,4),(3,4)\right>=9+16=25=\|\alpha\|^2. The length of a vector \alpha, also called the norm of \alpha, will be denoted as \|\alpha\|. This motivates the definition:

    \[\|\alpha\|=\sqrt{\left<\alpha,\alpha\right>}.\]

Assume now that we are given three vectors p,q and r forming a triangle. So r=p-q. Let \theta be the angle between p i q. The law of cosines states that:

    \[\|r\|^2=\|p\|^2+\|q\|^2-2\|p\|\|q\|\cos\theta.\]

Therefore:

    \[\left<p-q,p-q\right>=\left<p,p\right>+\left<q,q\right>-2\|p\|\|q\|\cos\theta,\]

So:

    \[\left<p,p\right>+\left<q,q\right>-2\left<p,q\right>=\left<p,p\right>+\left<q,q\right>-2\|p\|\|q\|\cos\theta,\]

So cosine of an angle between vectors is defined by the following formula:

    \[\cos\theta=\frac{\left <p,q\right>}{\|p\|\|q\|}.\]

One more application of scalar product is calculating perpendicular projection of a vector onto a direction given by a second vector. Let r be the perpendicular projection of p onto direction given by q. It will have the same direction as q and length \|p\|\cos \theta, where \theta is the angle between p and q. Therefore:

    \[r=\|p\|\cos \theta \cdot \frac{q}{\|q\|}=q\cdot \frac{\left<p,q\right>}{\left<q,q\right>},\]

because: \frac{q}{\|q\|} is the vector of length 1 in direction of q.

Properties of norm and product

It is not very hard to prove the following

  • \|u+v\|^2=\|u\|^2+\|v\|^2+2\langle u,v\rangle,
  • \|au\|=|a|\|u\|,
  • \|u\|=0 if and only if u is the zero vector
  • |\langle u,v\rangle|\leq \|u\|\|v\| (Schwarz inequality)
  • \|u+v\|\leq \|u\|+\|v\| (triangle inequality)
  • jeśli \langle u,v\rangle=0, to \|u+v\|^2=\|u\|^2+\|v\|^2 (Pitagoras Theorem)

Perpendicularity, perpendicular spaces

We know that two vectors v,w are perpendicular if cosine of the angle between them equals zero. Therefore v\bot w if and only if \left<v,w\right>=0.

Notice that if we would like to find all vectors w perpendicular to v, then the above is the equation we have to solve. Moreover this is a linear uniform equation. If we would like to find the set of vectors perpendicular to all vectors from a given list, then we will get a system of uniform linear equations. So given a linear subspace V, the set V^\bot (called the orthogonal complement of V) of all vectors perpendicular to all the vectors from V is also a linear subspace! It is the space of solutions of some system of linear equations.

For example, let V=lin((1,1,0,-1),(-1,0,2,0)) in the space with standard product. A vector (x,y,z,t) is perpendicular to those vectors (and so also to every vector of V), if \left<(1,1,0,-1), (x,y,z,t)\right>=0 and \left<(-1,0,2,0), (x,y,z,t)\right>=0, in other words, if it satisfies the following system of equations:

    \[\begin{cases}x+y-t=0\\-x+2z=0\end{cases},\]

so:

    \[\left[\begin{array}{cccc|c}1&1&0&-1&0\\-1&0&2&0&0\end{array}\right]\underrightarrow{w_2+w_1} \left[\begin{array}{cccc|c}1&1&0&-1&0\\0&1&2&-1&0\end{array}\right]\underrightarrow{w_1-w_2} \left[\begin{array}{cccc|c}1&0&-2&0&0\\0&1&2&-1&0\end{array}\right]\]

So the general solution has the following form: (2z,-2z+t,z,t), and therefore we have the following basis of V^\bot: (2,-2,1,0),(0,1,0,1).

Notice that the coefficients in a system of equations describing given linear space are vectors which span the perpendicular space! Which is a new insight into our method of finding a system of equations for a space given by its spanning vectors.

Orthogonal and orthonormal bases

A basis of a space will be called orthogonal, if every pair of vectors in it are perpendicular. E.g. (1,0,1),(-1,1,1),(-1,-2,1) is an orthogonal basis of \mathbb{R}^3 with standard product — indeed it is easy to check, that all pairs are perpendicular, e.g. \left<(-1,1,1),(-1,-2,1)\right>=1-2+1=0.

We will learn how to find such basis later on. Now let us notice that calculating coordinates of a vector in such a basis is fairly simple. Actually, we are calculating its projections onto vectors from the basis. Given vector v, its n-th coordinate in an orthogonal basis, which has b_n as its n-th vector is simply \frac{\left<v,b_n\right>}{\left<b_n,b_n\right>}. Therefore, coordinates of (1,0,-2) in the above exemplary basis are: \frac{\left<(1,0,-2),(1,0,1)\right>}{\left<(1,0,1),(1,0,1)\right>}=\frac{-1}{2}, \frac{-3}{3}=-1 and \frac{-3}{6}=\frac{-1}{2}.

Orthonormal basis is an orthogonal basis in which additionally all the vectors have length 1. Given an orthogonal basis we can simply divide each of the vectors by its length to get an orthonormal basis. So \frac{(1,0,1)}{\sqrt{2}}, \frac{(-1,1,1)}{\sqrt{3}}, \frac{(-1,2,1)}{\sqrt{6}} is an orthonormal basis of \mathbb{R}^3. Notice that calculating the coordinates of a vector in an orthonormal basis is even simpler. Since \left<b_n,b_n\right>=1, we get that the n-th coordinate of v is simply \left<v,b_n\right>, where b_n is the n-th vector of the basis. Therefore the coordinates of (1,0,-2) in the basis in the example are: \frac{-1}{\sqrt{2}}, \frac{-3}{\sqrt{3}} and \frac{-3}{\sqrt{6}}.

Projection onto a linear subspace

We already know how to calculate a projection of a vector onto a line defined by a given vector. To calculate a projection onto more-dimensional subspace V we have to calculate its orthogonal basis and calculate projections onto directions defined by each of the vectors from the basis and sum those projections.

Notice also that if r is a projection of a vector v onto V, and r' is its projection onto V^{\bot}, then r=v-r'. Which can be nicely used when calculating a projection of a vector onto a plane in the case of three-dimensional space. Indeed, if V is a plane in \mathbb{R}^3, then the perpendicular space is a line and is given by a vector. Denote this vector by n. Therefore the projection of v onto plane V is:

    \[r=v-\frac{\left<v,n\right>}{\left<n,n\right>}n.\]

Orthogonal reflection across a linear subspace

We can use our ability to calculate the projection r of v onto V, to easily calculate (see the image below) its image v' under reflection across V:

    \[v'=2r-v\]

symetria

Gram-Schmidt orthogonalization

Assume that we are given a space spanned by some vectors , e.g. lin((1,0,1,0),(0,1,-1,1),(0,0,0,1))=lin(v_1,v_2,v_3). We would like to find an orthogonal basis of this space. The method of finding such a basis is called Gram-Schmidt orthogonalization. The idea is to take as the first vector of the new basis the first vector from the original basis:

    \[w_1=v_1.\]

The second vector has to be be perpendicular to the first one, so we take the second vector and subtract its projection onto the first one. So:

    \[w_2=v_2-\frac{\left<w_1,v_2\right>}{\left<w_1,w_1\right>}w_1\]

so only perpendicular ,,part” is left. In the case of the third vector we need to subtract projections onto both already constructed vectors:

    \[w_3=v_3-\frac{\left<w_1,v_3\right>}{\left<w_1,w_1\right>}w_1-\frac{\left<w_2,v_3\right>}{\left<w_2,w_2\right>}w_2.\]

In the case of more dimensions we will continue this procedure further.

In our case in the space with standard product:

    \[w_1=(1,0,1,0).\]

    \[w_2=(0,1,-1,1)-\frac{\left<(1,0,1,0),(0,1,-1,1)\right>}{\left<(1,0,1,0),(1,0,1,0)\right>}(1,0,1,0)=\]

    \[=(0,1,-1,1)-\frac{-1}{2}(1,0,1,0)=\frac{1}{2}(1,2,-1,2).\]

    \[w_3=(0,0,0,1)-\frac{\left<(1,0,1,0),(0,0,0,1)\right>}{\left<(1,0,1,0),(1,0,1,0)\right>}(1,0,1,0)+\]

    \[-\frac{\left<(1,2,-1,2),(0,0,0,1)\right>}{\left<(1,2,-1,2),(1,2,-1,2)\right>}(1,2,-1,2)=\]

    \[=(0,0,0,1)-\frac{0}{2}(1,0,1,0)-\frac{2}{10}(1,2,-1,2)=\frac{1}{5}(-1,-2,1,3).\]

So the basis we were looking for is (1,0,1,0),(1,2,-1,2),(-1,-2,1,3) (I can drop the fractions since multiplication by a number does not change angles).

Linear maps: projection onto a linear subspace and reflection across a linear subspace

Notice that a projection onto a linear subspace V and reflection across V are linear mappings. Moreover, it is easy to see its eigenvectors:

  • since projection does not change vectors in V, they are eigenvectors with eigenvalue 1. On the other hand, vectors from V^\bot are multiplied by zero, so they are eigenvectors with eigenvalue zero.
  • since reflection does not change vectors in V, they are eigenvectors with eigenvalue 1. On the other hand, vectors from V^\bot are multiplied by -1, so they are eigenvectors with eigenvalue -1.

So basis consisting of vectors from a basis of V and of vectors from a basis of V^\bot is a basis of eigenvectors of both those maps. Which make it possible to calculate their formulas.

E.g. let V=\lin((1,0,1),(0,1,-1)) in the space with standard product. Therefore basis V^\bot is \{(-1,1,1)\}. So if \phi is the projection onto V, and \psi is the reflection across V, then (1,0,1),(0,1,-1) are eigenvectors with eigenvalue 1 of both maps. Also (-1,1,1) is an eigenvector with eigenvalue zero for \phi, and -1 for \psi. Therefor basis \mathcal{A}=\{(1,0,1),(0,1,-1),(-1,1,1)\} is a basis of eigenvectors of both maps, and::

    \[M(\phi)_{\mathcal{A}}^{\mathcal{A}}=\left[\begin{array}{ccc}1&0&0\\0&1&0\\0&0&0\end{array}\right]\]

    \[M(\psi)_{\mathcal{A}}^{\mathcal{A}}=\left[\begin{array}{ccc}1&0&0\\0&1&0\\0&0&-1\end{array}\right]\]

Let us calculate their formulas. We have:

    \[M(id)_{st}^{\mathcal{A}}=\left[\begin{array}{ccc}1&0&-1\\0&1&1\\1&-1&1\end{array}\right]\]

So:

    \[M(id)_{\mathcal{A}}^{st}=\left(M(id)_{\mathcal{A}}^{st}\right)^{-1}=\frac{1}{3}\left[\begin{array}{ccc}2&1&1\\1&2&-1\\-1&1&1\end{array}\right],\]

Therefore:

    \[M(\phi)_{st}^{st}=M(id)_{\mathcal{A}}^{st}M(\phi)_{\mathcal{A}}^{\mathcal{A}}M(id)^{\mathcal{A}}_{st}=\frac{1}{3}\left[\begin{array}{ccc}2&1&1\\1&2&-1\\1&-1&2\end{array}\right]\]

and

    \[M(\psi)_{st}^{st}=M(id)_{\mathcal{A}}^{st}M(\psi)_{\mathcal{A}}^{\mathcal{A}}M(id)^{\mathcal{A}}_{st}=\frac{1}{3}\left[\begin{array}{ccc}1&2&2\\2&1&-2\\2&-2&1\end{array}\right]\]

so:

    \[\phi((x,y,z))=\frac{1}{3}(2x+y+z,x+2y-z,x-y+2z)\]

and

    \[\psi((x,y,z))=\frac{1}{3}(x+2y+2z,2x+y-2z,2x-2y+z).\]

Gram’s determinant

Given a system of vectors v_1,\ldots, v_k in an euclidean space, the matrix G(v_1,\ldots , v_k)\in M_{k\times k}(\mathbb{R})=[\langle v_i,v_j\rangle]_{i,j\leq k} is called the Gram’s matrix of this system of vectors and its determinant is called Gram’s determinant W(v_1,\ldots, v_k). Immediately on can notice, that Gram’s matrix is symmetrical.

Notice in particular, that if the columns of a matrix A contain the coordinates of those vectors in a othonormal basis, then G(v_1,\ldots , v_k)=A^TA. Thus, if the number of vectors k equals the dimension of the whole space (i.e. if A is a square matrix), then W(v_1,\ldots , v_k)=(\det A)^2.

In particular, always W(v_1,\ldots , v_k)\geq 0. And it is equal to zero if and only if the system of vectors is linearly dependent.

Generalized products and Sylvester’s criterion

A generalized inner product of \alpha,\beta, is a function \left<\alpha,\beta\right>, such that:

  • \langle au+bv,w\rangle=a\langle u,w\rangle +b\langle v,w\rangle,
  • \langle u,v\rangle=\langle v,u\rangle.

(it is not necessarily positive definite).

Obviously, also for generalized inner product we can consider Gram matrices. In particular, let A the Gram’s matrix of a basis. We can then check whether the considered generalized product is an inner product.

Sylvester’s criterion determines whether a generalized product is positively definite or negatively definite.

How does it work? We study determinants of minors: let A^{(k)} be the matrix of size k\times k in the left upper corner of the matrix of a form we study. Let n\times n be the size of the matrix. Sylvester’s criterion states that if for any k\leq n we have \det A^{(k)} > 0, the product is positively definite and thus is a inner product.

E.g. let \langle (x,y),(a,b)\rangle =2xa+yb-xb-ya, so its matrix: \left[\begin{array}{cc}2&-1\\-1&1\end{array}\right], so A^{(1)}=[2] and A^{(2)}=\left[\begin{array}{cc}2&-1\\-1&1\end{array}\right], therefore \det A^{(1}=2>0 and \det A^{(2)}=2-1=1>0, so form q is positively definite.