3. Norms, inner products and balls

Norma

A norm is a way of measuring lengths of vectors (in the space \mathbb{R}^n). The most standard example is the Euclidean norm, which on the plane is given by the formula:

    \[\|(x,y)\|=\sqrt{x^2+y^2},\]

and generally

    \[\|(x_1,\ldots , x_n)\|=\sqrt{x_1^2+\ldots +x_n^2}.\]

But this is not the only way of measuring length. E.g. if we can move only along vertical and horizontal lines we may argue that it makes sense to define a norm as

    \[\|(x,y)\|=\|x\|+\|y\|\]

which is called the city norm (assuming, that the city has a perpendicular grid of streets, which we are moving along).
Yet another idea is

    \[\|(x,y)\|=\max(\|x\|,\|y\|),\]

which is so called maximum norm.

What do we expect from a norm? When a function which assigns a non-negative number to any vector can be called a norm? The definition follows. Such a function is a norm if it satisfies the following conditions.

  • \|u\|=0 if and only if u is the zero vector,
  • is a is a scalar, then \|au\|=|a|\cdot \|u\|,
  • \|u+v\|\leq \|u\|+\|v\| (triangle inequality).

Indeed, the city norm satisfies this definition, because \|(x,y)\|=0 if |x|+|y|=0, which means that x=y=0,

    \[\|a(x,y)\|=\|(ax,ay)\|=|ax|+|ay|=|a|(|x|+|y|)=|a|\|(x,y)\|,\]

and also

    \[\|(x,y)+(x',y')\|=\|(x+x',y,y')\|=\|x+x'|+|y+y'|\leq |x|+|x'|+|y|+|y'|=\|(x,y)\|+\|(x',y')\|.\]

Inner products

Norms can be created using inner products.

The typical example of an inner (scalar) product is the Euclidean (standard) product defined by

    \[\langle (x,y),(x',y')\rangle= xx'+yy',\]

and more generally in \R^n as

    \[\langle (x_1,\ldots, x_n),(x_1',\ldots x'_n)\rangle= x_1x_1'+\ldots +x_nx_n'.\]

But similarly as in the case of norms we can consider different inner products, e.g.

    \[\langle (x,y),(x',y')\rangle= 2xx'+yy'+xy'+x'y.\]

Generally, a function assigning two vectors a real number is called an inner product if it satisfies the following conditions.

  • \langle u+v,w\rangle=\langle u,w\rangle+\langle v,w\rangle,
  • \langle au,v\rangle = a\langle u,v\rangle, for a scalar a,
  • \langle u,v\rangle = \langle v,u\rangle,
  • \langle u,u\rangle\geq 0 and equals only if u is the zero vector.

Every inner product generates a norm given by

    \[\|u\|=\sqrt{\langle u,u\rangle}.\]

Schwartz inequality

Schwartz inequality is very useful when proving some facts related to norms and inner products. It states that for any x_1,\ldots, x_n, x_1',\ldots, x_n'

    \[(x_1x'_1+\ldots +x_nx'_n)^2\leq (x_1^2+\ldots +x_n^2)((x'_1)^2+\ldots +(x'_n)^2).\]

This inequality is easy to prove. After multiplication of the brackets and moving everything to the right-hand side, we get that 0\leq than a sum of squares of some differences.

In a more general form for any inner product and a norm generated by it:

    \[\langle u,v\rangle \leq \|u\|\|v\|.\]

Balls

A ball with radius r and center in zero with respect to a given norm is the set

    \[B_r=\{x\in \mathbb{R}^n\colon \|x\|<r\}.\]

A ball can look differently depending on the chosen norm. E.g. the unitary ball with respect to the Euclidean norm on the plate \{(x,y)\colon \sqrt{x^2+y^2}<1\} is a circle (red), but the unitary ball with respect to the city norm on the plane \{(x,y)\colon \|x\|+\|y\|<1\} is a square (blue).

Convex sets

A set (on the plane or more generally in \R^n) is convex, if for any two points in this set, all the points on the line segment between those points is in the set as well.

In analytical language we can formulate this property in the following way. A set is convex if for any point x,y in it and any t\in[0,1], the point tx+(1-t)y is in the set as well.