13. Scalar product and perpendicularity

Scalar product

To study angles between vectors it will be convenient to use scalar product. The scalar product of two vectors \alpha,\beta, is \left<\alpha,\beta\right>. Standard scalar product in \mathbb{R}^n is sum of products on subsequent places, so e.g.: \left<(1,2,-1),(2,0,1)\right>=1\cdot 2+2\cdot 0+(-1)\cdot 1=2+0-1=1.

Length of a vector and angles between vectors

By Pitagoras Theorem it is easy to see that \left<\alpha,\alpha\right> is the square of the length of a vector, e.g. \left<(3,4),(3,4)\right>=9+16=25=|\alpha|^2. The length of a vector \alpha, also called the norm of \alpha, will be denoted as |\alpha|. We get that:


Assume now that we are given three vectors p,q and r forming a triangle. So r=p-q. Let \theta be the angle between p i q. The law of cosines states that:



    \[\left <p-q,p-q\right>=\left <p,p\right>+\left<q,q\right>-2|p||q|\cos\theta,\]


    \[\left <p,p\right>+\left<q,q\right>-2\left <p,q\right>=\left <p,p\right>+\left<q,q\right>-2|p||q|\cos\theta,\]

So cosine of an angle between vectors is given by the following formula:

    \[\cos\theta=\frac{\left <p,q\right>}{|p||q|}.\]

One more application of scalar product is calculating perpendicular projection of a vector onto a direction given by a second vector. Let r be the perpendicular projection of p onto direction given by q. It will have the same direction as q and length |p|\cos \theta, where \theta is the angle between p and q. Therefore:

    \[r=|p|\cos \theta \cdot \frac{q}{|q|}=q\cdot \frac{\left <p,q\right>}{\left<q,q\right>},\]

because: \frac{q}{|q|} is the vector of length 1 in direction of q.

Perpendicularity, perpendicular spaces

We know that two vectors v,w are perpendicular if cosine of the angle between them equals zero. Therefore v\bot w if and only if \left<v,w\right>=0.

Notice that if we would like to find all vectors w perpendicular to v, then the above is the equation we have to solve. Moreover this is a linear uniform equation. If we would like to find the set of vectors perpendicular to all vectors from a given list, then we will get a system of uniform linear equations. So given a linear subspace V, the set V^\bot (called the orthogonal complement of V) of all vectors perpendicular to all the vectors from V is also a linear subspace! It is the space of solutions of some system of linear equations.

For example, let V=lin((1,1,0,-1),(-1,0,2,0)). A vector (x,y,z,t) is perpendicular to those vectors (and so also to every vector of V), if \left<(1,1,0,-1), (x,y,z,t)\right>=0 and \left<(-1,0,2,0), (x,y,z,t)\right>=0, in other words, if it satisfies the following system of equations:



    \[\left[\begin{array}{cccc|c}1&1&0&-1&0\\-1&0&2&0&0\end{array}\right]\underrightarrow{w_2+w_1} \left[\begin{array}{cccc|c}1&1&0&-1&0\\0&1&2&-1&0\end{array}\right]\underrightarrow{w_1-w_2} \left[\begin{array}{cccc|c}1&0&-2&0&0\\0&1&2&-1&0\end{array}\right]\]

So the general solution has the following form: (2z,-2z+t,z,t), and therefore we have the following basis of V^\bot: (2,-2,1,0),(0,1,0,1).

Notice that the coefficients in a system of equations describing given linear space are vectors which span the perpendicular space! Which is a new insight into our method of finding a system of equations for a space given by its spanning vectors.

Orthogonal and orthonormal bases

A basis of a space will be called orthogonal, if every pair of vectors in it are perpendicular. E.g. (1,0,1),(-1,1,1),(-1,-2,1) is an orthogonal basis of \mathbb{R}^3 — indeed it is easy to check, that all pairs are perpendicular, e.g. \left<(-1,1,1),(-1,-2,1)\right>=1-2+1=0.

We will learn how to find such basis later on. Now let us notice that calculating coordinates of a vector in such a basis is fairly simple. Actually, we are calculating its projections onto vectors from the basis. Given vector v, its n-th coordinate in an orthogonal basis, which has b_n as its n-th vector is simply \frac{\left<v,b_n\right>}{\left<b_n,b_n\right>}. Therefore, coordinates of (1,0,-2) in the above exemplary basis are: \frac{\left<(1,0,-2),(1,0,1)\right>}{\left<(1,0,1),(1,0,1)\right>}=\frac{-1}{2}, \frac{-3}{3}=-1 and \frac{-3}{6}=\frac{-1}{2}.

Orthonormal basis is an orthogonal basis in which additionally all the vectors have length 1. Given an orthogonal basis we can simply divide each of the vectors by its length to get an orthonormal basis. So \frac{(1,0,1)}{\sqrt{2}}, \frac{(-1,1,1)}{\sqrt{3}}, \frac{(-1,2,1)}{\sqrt{6}} is an orthonormal basis of \mathbb{R}^3. Notice that calculating the coordinates of a vector in an orthonormal basis is even simpler. Since \left<b_n,b_n\right>=1, we get that the n-th coordinate of v is simply \left<v,b_n\right>, where b_n is the n-th vector of the basis. Therefore the coordinates of (1,0,-2) in the basis in the example are: \frac{-1}{\sqrt{2}}, \frac{-3}{\sqrt{3}} and \frac{-3}{\sqrt{6}}.

Projection onto a linear subspace

We already know how to calculate a projection of a vector onto a line defined by a given vector. To calculate a projection onto more-dimensional subspace V we have to calculate its orthogonal basis and calculate projections onto directions defined by each of the vectors from the basis and sum those projections.

Notice also that if r is a projection of a vector v onto V, and r' is its projection onto V^{\bot}, then r=v-r'. Which can be nicely used when calculating a projection of a vector onto a plane in the case of three-dimensional space. Indeed, if V is a plane in \mathbb{R}^3, then the perpendicular space is a line and is given by a vector. Denote this vector by n. Therefore the projection of v onto plane V is:


Orthogonal reflection across a linear subspace

We can use our ability to calculate the projection r of v onto V, to easily calculate (see the image below) its image v' under reflection across V:



Gram-Schmidt orthogonalization

Assume that we are given a space spanned by some vectors, e.g. lin((1,0,1,0),(0,1,-1,1),(0,0,0,1))=lin(v_1,v_2,v_3). We would like to find an orthogonal basis of this space. The method of finding such a basis is called Gram-Schmidt orthogonalization. The idea is to take as the first vector of the new basis the first vector from the original basis:


The second vector has to be be perpendicular to the first one, so we take the second vector and subtract its projection onto the first one. So:


so only perpendicular ,,part” is left. In the case of the third vector we need to subtract projections onto both already constructed vectors:


In the case of more dimensions we will continue this procedure further.

In our case:







So the basis we were looking for is (1,0,1,0),(1,2,-1,2),(-1,-2,1,3) (I can drop the fractions since multiplication by a number does not change angles).

Linear maps: projection onto a linear subspace and reflection across a linear subspace

Notice that a projection onto a linear subspace V and reflection across V are linear mappings. Moreover, it is easy to see its eigenvectors:

  • since projection does not change vectors in V, they are eigenvectors with eigenvalue 1. On the other hand, vectors from V^\bot are multiplied by zero, so they are eigenvectors with eigenvalue zero.
  • since reflection does not change vectors in V, they are eigenvectors with eigenvalue 1. On the other hand, vectors from V^\bot are multiplied by -1, so they are eigenvectors with eigenvalue -1.

So basis consisting of vectors from a basis of V and of vectors from a basis of V^\bot is a basis of eigenvectors of both those maps. Which make it possible to calculate their formulas.

E.g. let V=\lin((1,0,1),(0,1,-1)). Therefore basis V^\bot is \{(-1,1,1)\}. So if \phi is the projection onto V, and \psi is the reflection across V, then (1,0,1),(0,1,-1) are eigenvectors with eigenvalue 1 of both maps. Also (-1,1,1) is an eigenvector with eigenvalue zero for \phi, and -1 for \psi. Therefor basis \mathcal{A}=\{(1,0,1),(0,1,-1),(-1,1,1)\} is a basis of eigenvectors of both maps, and::



Let us calculate their formulas. We have: