Definition and Basic Properties

In ${\mathbb{R}}^2$ , given two vectors ${\mathbf x}=(x_1, x_2), \; {\mathbf y}=(y_1, y_2)$ , we know the inner product ${\mathbf x}\cdot {\mathbf y}= x_1 y_1 + x_2 y_2$ . Note that for any ${\mathbf x}, {\mathbf y}, {\mathbf z}\in {\mathbb{R}}^2$ and ${\alpha}\in {\mathbb{R}}$ , this inner product satisfies the conditions

$${\mathbf x}\cdot ({\mathbf y}+ {\alpha}{\mathbf z}) = {\mathbf x}... ...thbf y}\cdot {\mathbf x}, \; {\mbox{ and }} \; {\mathbf x}\cdot{\mathbf x}\ge 0$$

and ${\mathbf x}\cdot {\mathbf x}= 0$ if and only if ${\mathbf x}={\mathbf 0}$ . Thus, we are motivated to define an inner product on an arbitrary vector space.

DEFINITION 5.1.1 (Inner Product)   Let $V ({\mathbb{F}})$ be a vector space over ${\mathbb{F}}.$ An inner product over $V({\mathbb{F}}),$ denoted by $\langle \;, \; \rangle,$ is a map,

$$\langle \;, \; \rangle \; : V \times V \longrightarrow {\mathbb{F}}$$

such that for ${\mathbf u}, {\mathbf v}, {\mathbf w}\in V$ and $a, b \in {\mathbb{F}}$

1. $\langle a {\mathbf u}+ b {\mathbf v}, {\mathbf w}\rangle = a \langle {\mathbf u}, {\mathbf w}\rangle + b \langle {\mathbf v}, {\mathbf w}\rangle,$
2. $\langle {\mathbf u}, {\mathbf v}\rangle = {\overline{\langle {\mathbf v}, {\mathbf u}\rangle}},$ the complex conjugate of $\langle {\mathbf u}, {\mathbf v}\rangle,$ and
3. $\langle {\mathbf u}, {\mathbf u}\rangle \geq 0$ for all ${\mathbf u}\in V$ and equality holds if and only if ${\mathbf u}= {\mathbf 0}.$

DEFINITION 5.1.2 (Inner Product Space)   Let $V$ be a vector space with an inner product $\langle\; , \;\rangle.$ Then $(V, \langle\; , \;\rangle)$ is called an inner product space, in short denoted by IPS.

EXAMPLE 5.1.3   The first two examples given below are called the STANDARD INNER PRODUCT or the DOT PRODUCT on ${\mathbb{R}}^n$ and ${\mathbb{C}}^n,$ respectively..

1. Let $V = {\mathbb{R}}^n$ be the real vector space of dimension $n.$ Given two vectors ${\mathbf u}= (u_1, u_2, \ldots, u_n)$ and ${\mathbf v}=(v_1, v_2, \ldots, v_n)$ of $V,$ we define
   $$\langle u, v \rangle = u_1 v_1 + u_2 v_2 + \cdots + u_n v_n = {\mathbf u}{\mathbf v}^t.$$
   Verify $\langle \;, \; \rangle$ is an inner product.
2. Let $V= {\mathbb{C}}^n$ be a complex vector space of dimension $n.$ Then for ${\mathbf u}= (u_1, u_2, \ldots, u_n)$ and ${\mathbf v}=(v_1, v_2, \ldots, v_n)$ in $V,$ check that
   $$\langle u, v \rangle = u_1 {\overline{v_1}} + u_2 {\overline{v_2}} + \cdots + u_n {\overline{v_n}} = {\mathbf u}{\mathbf v}^*$$
   is an inner product.
3. Let $V = {\mathbb{R}}^2$ and let $A = \begin{bmatrix}4 & -1 \\ -1 & 2 \end{bmatrix}.$ Define $\langle {\mathbf x}, {\mathbf y}\rangle = {\mathbf x}A {\mathbf y}^{t}.$ Check that $\langle \;, \; \rangle$ is an inner product. Hint: Note that ${\mathbf x}A {\mathbf y}^{t} = 4 x_1 y_1 - x_1 y_2 - x_2 y_1 + 2 x_2 y_2.$
4. let ${\mathbf x}=(x_1,x_2,x_3), \; {\mathbf y}=(y_1,y_2,y_3) \in {\mathbb{R}}^3.,$ Show that $\langle {\mathbf x},{\mathbf y}\rangle = 10 x_1 y_1 + 3 x_1 y_2 + 3 x_2 y_1 + 2 x_2 y_2 + x_2 y_3 + x_3 y_2 + x_3 y_3$ is an inner product in ${\mathbb{R}}^3 ({\mathbb{R}}).$
5. Consider the real vector space ${\mathbb{R}}^2$ . In this example, we define three products that satisfy two conditions out of the three conditions for an inner product. Hence the three products are not inner products.
   1. Define $\langle {\mathbf x}, {\mathbf y}\rangle = \langle (x_1, x_2), (y_1, y_2) \rangle = x_1 y_1.$ Then it is easy to verify that the third condition is not valid whereas the first two conditions are valid.
   2. Define $\langle {\mathbf x}, {\mathbf y}\rangle = \langle (x_1, x_2), (y_1, y_2) \rangle = x_1^2 + y_1^2 + x_2^2 + y_2^2.$ Then it is easy to verify that the first condition is not valid whereas the second and third conditions are valid.
   3. Define $\langle {\mathbf x}, {\mathbf y}\rangle = \langle (x_1, x_2), (y_1, y_2) \rangle = x_1y_1^3 + x_2 y_2^3.$ Then it is easy to verify that the second condition is not valid whereas the first and third conditions are valid.

Remark 5.1.4   Note that in parts $1$ and $2$ of Example 5.1.3, the inner products are ${\mathbf u}{\mathbf v}^t$ and ${\mathbf u}{\mathbf v}^{*}$ , respectively. This occurs because the vectors ${\mathbf u}$ and ${\mathbf v}$ are row vectors. In general, ${\mathbf u}$ and ${\mathbf v}$ are taken as column vectors and hence one uses the notation ${\mathbf u}^t {\mathbf v}$ or ${\mathbf u}^*{\mathbf v}$ .

EXERCISE 5.1.5   Verify that inner products defined in parts $3$ and $4$ of Example 5.1.3, are indeed inner products.

DEFINITION 5.1.6 (Length/Norm of a Vector)   For ${\mathbf u}\in V,$ we define the length (norm) of ${\mathbf u},$ denoted $\Vert {\mathbf u}\Vert,$ by $\Vert {\mathbf u}\Vert = \sqrt{\langle {\mathbf u}, {\mathbf u}\rangle},$ the positive square root.

A very useful and a fundamental inequality concerning the inner product is due to Cauchy and Schwartz. The next theorem gives the statement and a proof of this inequality.

THEOREM 5.1.7 (Cauchy-Schwartz inequality)   Let $V ({\mathbb{F}})$ be an inner product space. Then for any ${\mathbf u}, {\mathbf v}\in V$

$$\vert \langle {\mathbf u}, {\mathbf v}\rangle \vert \leq \Vert {\mathbf u}\Vert \; \Vert {\mathbf v}\Vert.$$

The equality holds if and only if the vectors ${\mathbf u}$ and ${\mathbf v}$ are linearly dependent. Further, if ${\mathbf u}\neq {\mathbf 0}$ , then ${\mathbf v}= \displaystyle \langle {\mathbf v}, \frac{{\mathbf u}}{\Vert {\mathbf u}\Vert} \rangle \frac{{\mathbf u}}{\Vert{\mathbf u}\Vert}.$

Proof. If ${\mathbf u}= {\mathbf 0},$ then the inequality holds. Let ${\mathbf u}\neq {\mathbf 0}.$ Note that $\langle {\lambda}{\mathbf u}+ {\mathbf v}, {\lambda}{\mathbf u}+ {\mathbf v}\rangle \geq 0$ for all ${\lambda}\in {\mathbb{F}}.$ In particular, for ${\lambda}= \displaystyle - \frac{\langle {\mathbf v},{\mathbf u}\rangle}{\Vert {\mathbf u}\Vert^2},$ we get

$$\leq \langle {\lambda}{\mathbf u}+ {\mathbf v}, {\lambda}{\mathbf u}+ {\mathbf v}\rangle$$ 0

$$= {\lambda}\overline{{\lambda}} \Vert{\mathbf u}\Vert^2 + {\lambda}... ...ne{{\lambda}} \langle {\mathbf v}, {\mathbf u}\rangle + \Vert{\mathbf v}\Vert^2$$

$$= \frac{\langle {\mathbf v},{\mathbf u}\rangle}{\Vert {\mathbf u}\V... ...bf u} \Vert^2}\langle {\mathbf v}, {\mathbf u}\rangle + \Vert{\mathbf v}\Vert^2$$

$$= \Vert{\mathbf v}\Vert^2 - \frac{\vert\langle {\mathbf v},{\mathbf u}\rangle\vert^2}{\Vert {\mathbf u}\Vert^2}.$$

Or, in other words

$$\vert\langle {\mathbf v},{\mathbf u}\rangle\vert^2 \leq \Vert{\mathbf u}\Vert^2 \Vert{\mathbf v}\Vert^2$$

and the proof of the inequality is over.

Observe that if ${\mathbf u}\neq {\mathbf 0}$ then the equality holds if and only of ${\lambda}{\mathbf u}+ {\mathbf v}= {\mathbf 0}$ for ${\lambda}= - \displaystyle \frac{\langle {\mathbf v}, {\mathbf u}\rangle }{\Vert {\mathbf u}\Vert^2}.$ That is, ${\mathbf u}$ and ${\mathbf v}$ are linearly dependent. We leave it for the reader to prove

$${\mathbf v}= \displaystyle \langle {\mathbf v}, \frac{{\mathbf u}}{\Vert {\mathbf u}\Vert} \rangle \frac{{\mathbf u}}{\Vert{\mathbf u}\Vert}.$$

height6pt width 6pt depth 0pt

DEFINITION 5.1.8 (Angle between two vectors)   Let $V$ be a real vector space. Then for every ${\mathbf u}, {\mathbf v}\in V,$ by the Cauchy-Schwartz inequality, we have

$$-1 \leq \frac{\langle {\mathbf u}, {\mathbf v}\rangle}{\Vert{\mathbf u}\Vert \; \Vert{\mathbf v}\Vert} \leq 1.$$

We know that $\cos : [0, \pi] \longrightarrow [-1, \; 1]$ is an one-one and onto function. Therefore, for every real number $$\frac{\langle {\mathbf u}, {\mathbf v}\rangle}{\Vert{\mathbf u}\Vert \; \Vert{\mathbf v}\Vert},$$ there exists a unique $\theta, \; 0 \leq \theta \leq \pi,$ such that

$$\cos \theta = \frac{\langle {\mathbf u}, {\mathbf v}\rangle}{\Vert{\mathbf u}\Vert \; \Vert{\mathbf v}\Vert}.$$

1. The real number $\theta$ with $0 \leq \theta \leq \pi$ and satisfying $\cos \theta = \displaystyle\frac{\langle {\mathbf u}, {\mathbf v}\rangle}{\Vert{\mathbf u}\Vert \; \Vert{\mathbf v}\Vert}$ is called the angle between the two vectors ${\mathbf u}$ and ${\mathbf v}$ in $V.$
2. The vectors ${\mathbf u}$ and ${\mathbf v}$ in $V$ are said to be orthogonal if $\langle {\mathbf u}, {\mathbf v}\rangle = 0.$
3. A set of vectors $\{{\mathbf u}_1, {\mathbf u}_2, \ldots, {\mathbf u}_n\}$ is called mutually orthogonal if $\langle {\mathbf u}_i, {\mathbf u}_j \rangle = 0$ for all $1 \leq i \neq j \leq n.$

DEFINITION 5.1.9 (Orthogonal Complement)   Let $W$ be a subspace of a vector space $V$ with inner product $\langle \;, \; \rangle$ . Then the subspace

$$W^{\perp} = \{ {\mathbf v}\in V : \; \langle {\mathbf v}, {\mathbf w}\rangle = 0 \; {\mbox{ for all }} \;\; {\mathbf w}\in W\}$$

is called the orthogonal complement of $W$ in $V.$

EXERCISE 5.1.10  

1. Let $\{{\mathbf e}_1, {\mathbf e}_2, \ldots, {\mathbf e}_n\}$ be the standard basis of ${\mathbb{R}}^n.$ Then prove that with respect to the standard inner product on ${\mathbb{R}}^n,$ the vectors ${\mathbf e}_i$ satisfy the following:
   1. $\Vert {\mathbf e}_i\Vert = 1$ for $\; 1 \leq i \leq n.$
   2. $\langle {\mathbf e}_i, {\mathbf e}_j \rangle = 0$ for $1 \leq i \neq j \leq n.$
2. Recall the following inner product on ${\mathbb{R}}^2: \;$ for ${\mathbf x}= (x_1, x_2)^t$ and ${\mathbf y}= (y_1, y_2)^t,$
   $$\langle {\mathbf x}, {\mathbf y}\rangle = 4 x_1 y_1 - x_1 y_2 - x_2 y_1 + 2 x_2 y_2.$$
   1. Find the angle between the vectors $e_1 = (1,0)^t$ and $e_2 = (0,1)^t.$
   2. Let ${\mathbf u}= (1,0)^t.$ Find ${\mathbf v}\in {\mathbb{R}}^2$ such that $\langle {\mathbf v}, {\mathbf u} \rangle = 0.$
   3. Find two vectors ${\mathbf x}, {\mathbf y}\in {\mathbb{R}}^2,$ such that $\Vert{\mathbf x}\Vert = \Vert{\mathbf y}\Vert = 1$ and $\langle {\mathbf x}, {\mathbf y}\rangle = 0.$
3. Find an inner product in ${\mathbb{R}}^2$ such that the following conditions hold:
   $$\Vert (1, 2) \Vert = \Vert (2, -1) \Vert = 1, \;\; {\mbox{ and }} \;\; \langle (1, 2), \; (2, -1) \rangle = 0.$$
   [Hint: Consider a symmetric matrix $A = \begin{bmatrix}a & b \\ b & c \end{bmatrix}.$ Define $\langle {\mathbf x}, {\mathbf y}\rangle = {\mathbf y}^t A {\mathbf x}$ and solve a system of $3$ equations for the unknowns $a, b, c$ .]
4. Let $W = \{ (x,y,z) \in {\mathbb{R}}^3: x + y + z = 0 \}.$ Find $W^{\perp}$ with respect to the standard inner product.
5. Let $W$ be a subspace of a finite dimensional inner product space $V$ . Prove that $(W^{\perp})^{\perp} = W.$
6. Let $V$ be a complex vector space with $\dim (V) = n.$ Fix an ordered basis ${\cal B}= ( {\mathbf u}_1, {\mathbf u}_2, \ldots, {\mathbf u}_n).$ Define a map
   $$\langle\;, \; \rangle \; : V \times V \longrightarrow {\mathbb{C}... ...}} \; \langle {\mathbf u}, {\mathbf v}\rangle = \sum_{i=1}^n a_i \overline{b_i}$$
   whenever $[{\mathbf u}]_{{\cal B}} = (a_1, a_2, \ldots, a_n)^t$ and $[{\mathbf v}]_{{\cal B}} = (b_1, b_2, \ldots, b_n)^t.$ Show that the above defined map is indeed an inner product.
7. Let ${\mathbf x}=(x_1,x_2,x_3), \; {\mathbf y}=(y_1,y_2,y_3) \in {\mathbb{R}}^3.$ Show that
   $$\langle {\mathbf x},{\mathbf y}\rangle = 10 x_1 y_1 + 3 x_1 y_2 + 3 x_2 y_1 + 2 x_2 y_2 + x_2 y_3 + x_3 y_2 + x_3 y_3$$
   is an inner product in ${\mathbb{R}}^3 ({\mathbb{R}}).$ With respect to this inner product, find the angle between the vectors $(1,1,1)$ and $(2, -5, 2).$
8. Consider the set $M_{n \times n}({\mathbb{R}})$ of all real square matrices of order $n.$ For $A, B \in M_{n \times n}({\mathbb{R}})$ we define $\langle A, B \rangle = tr(A B^{t}).$ Then
   $$\langle A + B, C\rangle = tr\bigl( (A+B)C^t \bigr) = tr(A C^t) + tr(B C^t) = \langle A , C\rangle + \langle B, C\rangle.$$
   $$\langle A, B \rangle = tr(A B^t) = tr( \; (A B^t)^t\;) = tr(B A^t) = \langle B, A \rangle.$$
   Let $A = (a_{ij}).$ Then
   $$\langle A, A \rangle = tr(A A^t) = \sum_{i=1}^n (A A^t)_{ii} = \sum_{i=1}^n \sum_{j=1}^n a_{ij} a_{ij} = \sum_{i=1}^n \sum_{j=1}^n a_{ij}^2$$
   and therefore, $\langle A, A \rangle > 0$ for all non-zero matrices $A.$ So, it is clear that $\langle A, B \rangle$ is an inner product on $M_{n \times n}({\mathbb{R}})$ .

   If $W = \{ A \in M_{n \times n}({\mathbb{R}}) : \; A^t = A\}$ , determine $W^{\perp}.$

9. Let $V$ be the real vector space of all continuous functions with domain $[-2 \pi, 2 \pi].$ That is, $V = C[-2 \pi, \; 2 \pi].$ Then show that $V$ is an inner product space with inner product $\int_{-1}^1 f(x) g(x) dx.$

   For different values of $m$ and $n,$ find the angle between the functions $\cos (m x)$ and $\sin (n x).$

10. Let $V$ be an inner product space. Prove that
    $$\Vert {\mathbf u}+ {\mathbf v}\Vert \leq \Vert {\mathbf u}\Vert +... ... {\mathbf v}\Vert \;\; {\mbox{ for every }} \;\; {\mathbf u}, {\mathbf v}\in V.$$
    This inequality is called the TRIANGLE INEQUALITY.
11. Let $z_1, z_2, \ldots, z_n \in {\mathbb{C}}.$ Use the Cauchy-Schwartz inequality to prove that
    $$\vert z_1 + z_2 + \cdots + z_n\vert \leq \sqrt{n (\vert z_1\vert^2 + \vert z_2\vert^2 + \cdots + \vert z_n\vert^2)}.$$
    When does the equality hold?
12. Let ${\mathbf x}, {\mathbf y}\in {\mathbb{R}}^n.$ Observe that $\langle {\mathbf x}, {\mathbf y}\rangle = \langle {\mathbf y}, {\mathbf x}\rangle.$ Hence or otherwise prove the following:
    1. $\langle {\mathbf x}, {\mathbf y}\rangle = 0 \Longleftrightarrow \Vert {\mathbf x}- {\mathbf y}\Vert^2 = \Vert{\mathbf x}\Vert^2 + \Vert{\mathbf y}\Vert^2, \;\;$ (This is called PYTHAGORAS THEOREM).
    2. $\Vert {\mathbf x}\Vert= \Vert {\mathbf y}\Vert \Longleftrightarrow \langle {\mathbf x}+ {\mathbf y}, {\mathbf x}-{\mathbf y}\rangle = 0, \;\;$ ( ${\mathbf x}$ and ${\mathbf y}$ form adjacent sides of a rhombus as the diagonals ${\mathbf x}+ {\mathbf y}$ and ${\mathbf x}- {\mathbf y}$ are orthogonal).
    3. $\Vert{\mathbf x}+ {\mathbf y}\Vert^2 + \Vert {\mathbf x}- {\mathbf y}\Vert^2 = 2 \Vert{\mathbf x}\Vert^2 + 2 \Vert{\mathbf y}\Vert^2, \;\;$ (This is called the PARALLELOGRAM LAW).
    4. $4 \langle {\mathbf x}, {\mathbf y}\rangle = \Vert {\mathbf x}+ {\mathbf y}\Vert^2 - \Vert {\mathbf x}- {\mathbf y}\Vert^2$ (This is called the POLARISATION IDENTITY).
       Remark 5.1.11  

       1. Suppose the norm of a vector is given. Then, the polarisation identity can be used to define an inner product.
       2. Observe that if $\langle {\mathbf x}, {\mathbf y}\rangle = 0$ then the parallelogram spanned by the vectors ${\mathbf x}$ and ${\mathbf y}$ is a rectangle. The above equality tells us that the lengths of the two diagonals are equal.
       
       
       

       Are these results true if ${\mathbf x},{\mathbf y}\in {\mathbb{C}}^n ({\mathbb {C}})?$

13. Let ${\mathbf x}, {\mathbf y}\in {\mathbb{C}}^n ({\mathbb{C}}).$ Prove that
    1. $4 \langle {\mathbf x},{\mathbf y}\rangle = \Vert{\mathbf x}+ {\mathbf y}\Vert^... ...ert{\mathbf x}+ i{\mathbf y}\Vert^2 - i \Vert{\mathbf x}- i {\mathbf y}\Vert^2.$
    2. If ${\mathbf x}\neq {\mathbf 0}$ then $\;\; \Vert{\mathbf x}+ i {\mathbf x}\Vert^2 = \Vert{\mathbf x}\Vert^2 + \Vert i {\mathbf x}\Vert^2,$ even though $\langle {\mathbf x}, i {\mathbf x}\rangle \neq 0.$
    3. If $\Vert{\mathbf x}+ {\mathbf y}\Vert^2 = \Vert{\mathbf x}\Vert^2 + \Vert{\mathbf y}\Vert^2$ and $\Vert{\mathbf x}+ i {\mathbf y}\Vert^2 = \Vert{\mathbf x}\Vert^2 + \Vert i {\mathbf y}\Vert^2$ then show that $\langle {\mathbf x}, {\mathbf y}\rangle = 0.$
14. Let $\langle \;, \; \rangle$ denote the standard inner product on ${\mathbb{C}}^n({\mathbb{C}})$ and let $A$ be an $n \times n$ matrix. Then prove that $\langle {\mathbf x}, {\mathbf y}\rangle = {\mathbf x}{\mathbf y}^*$ and $\langle {\mathbf x}A, {\mathbf y}\rangle = \langle {\mathbf x}, {\mathbf y}A^* \rangle$ for all ${\mathbf x}, {\mathbf y}\in {\mathbb{C}}^n$ .

    Or equivalently, if ${\mathbf u}$ and ${\mathbf v}$ are column vectors then $\langle A {\mathbf u}, {\mathbf v}\rangle = \langle {\mathbf u}, A^* {\mathbf v}\rangle$ .

15. Let $V$ be an $n$ -dimensional inner product space, with an inner product $\langle\; , \;\rangle.$ Let ${\mathbf u}\in V$ be a fixed vector with $\Vert {\mathbf u}\Vert = 1.$ Then give reasons for the following statements.
    1. Let $S^{\perp} = \{ {\mathbf v}\in V \;: \; \langle {\mathbf v}, {\mathbf u}\rangle = 0 \}.$ Then $S$ is a subspace of $V$ of dimension $n-1.$
    2. Let $0 \neq {\alpha}\in {\mathbb{F}}$ and let $S = \{ {\mathbf v}\in V \;: \; \langle {\mathbf v}, {\mathbf u}\rangle={\alpha}\}.$ Then $S$ is not a subspace of $V.$
    3. For any ${\mathbf v}\in S,$ there exists a vector ${\mathbf v}_0 \in S^{\perp},$ such that ${\mathbf v}= {\mathbf v}_0 + {\alpha}{\mathbf u}.$

THEOREM 5.1.12   Let $V$ be an inner product space. Let $\{{\mathbf u}_1, {\mathbf u}_2, \ldots, {\mathbf u}_n\}$ be a set of non-zero, mutually orthogonal vectors of $V.$

1. Then the set $\{{\mathbf u}_1, {\mathbf u}_2, \ldots, {\mathbf u}_n\}$ is linearly independent.
2. $\Vert \sum\limits_{i=1}^n \alpha_i {\mathbf u}_i \Vert^2 = \sum\limits_{i=1}^n \vert\alpha_i\vert^2 \Vert {\mathbf u}_i \Vert^2;$
3. Let $\dim(V) = n$ and also let $\Vert{\mathbf u}_i\Vert = 1$ for $i=1,2,\ldots,n.$ Then for any ${\mathbf v}\in V,$
   $${\mathbf v}= \sum\limits_{i=1}^n \langle {\mathbf v}, {\mathbf u}_i \rangle {\mathbf u}_i.$$
   In particular, $\langle {\mathbf v}, {\mathbf u}_i \rangle = 0$ for all $i = 1, 2, \ldots, n$ if and only if ${\mathbf v}= {\mathbf 0}.$

Proof. Consider the set of non-zero, mutually orthogonal vectors $\{{\mathbf u}_1, {\mathbf u}_2, \ldots, {\mathbf u}_n\}.$ Suppose there exist scalars $c_1, c_2, \ldots, c_n$ not all zero, such that

$$c_1 {\mathbf u}_1 + c_2 {\mathbf u}_2 + \cdots + c_n {\mathbf u}_n = {\mathbf 0}.$$

Then for $1 \leq i \leq n,$ we have

$$0 = \langle {\mathbf 0}, {\mathbf u}_i \rangle = \langle c_1 {\ma... ..._i \rangle = \sum_{j=1}^n c_j \langle {\mathbf u}_j, {\mathbf u}_i\rangle = c_i$$

as $\langle {\mathbf u}_j, {\mathbf u}_i\rangle = 0$ for all $j \neq i$ and $\langle {\mathbf u}_i, {\mathbf u}_i\rangle = 1.$ This gives a contradiction to our assumption that some of the $c_i$ 's are non-zero. This establishes the linear independence of a set of non-zero, mutually orthogonal vectors.

For the second part, using $$\langle {\mathbf u}_i, {\mathbf u}_j \rangle = \left\{ \begin... ... {\mathbf u}_i\Vert^2 & {\mbox{ if }} i = j \end{array} \right.$$ for $1 \leq i, j \leq n,$ we have

$$\Vert\sum_{i=1}^n {\alpha}_i {\mathbf u}_i \Vert^2 = \langle \sum_{i=1}^n {\alpha}_i {\mathbf u}_i, \sum_{i=1}^n {\alp... ...{\alpha}_i \langle {\mathbf u}_i, \sum_{j=1}^n {\alpha}_j {\mathbf u}_j \rangle$$

$$= \sum_{i=1}^n {\alpha}_i \sum_{j=1}^n \overline{{\alpha}_j} \langl... ...n {\alpha}_i \overline{{\alpha}_i} \langle {\mathbf u}_i, {\mathbf u}_i \rangle$$

$$= \sum_{i=1}^n \vert{\alpha}_i\vert^2 \Vert {\mathbf u}_i\Vert^2.$$

For the third part, observe from the first part, the linear independence of the non-zero mutually orthogonal vectors ${\mathbf u}_1, {\mathbf u}_2, \ldots, {\mathbf u}_n.$ Since $\dim(V) = n,$ they form a basis of $V.$ Thus, for every vector ${\mathbf v}\in V,$ there exist scalars ${\alpha}_i, \; 1 \leq i \leq n,$ such that ${\mathbf v}= \sum_{i=1}^n {\alpha}_i {\mathbf u}_n.$ Hence,

$$\langle {\mathbf v}, {\mathbf u}_j \rangle = \langle \sum_{i=1}^n... ...m_{i=1}^n {\alpha}_i \langle {\mathbf u}_i, {\mathbf u}_j \rangle = {\alpha}_j.$$

Therefore, we have obtained the required result. height6pt width 6pt depth 0pt

DEFINITION 5.1.13 (Orthonormal Set)   Let $V$ be an inner product space. A set of non-zero, mutually orthogonal vectors $\{{\mathbf v}_1, {\mathbf v}_2, \ldots, {\mathbf v}_n \}$ in $V$ is called an orthonormal set if $\Vert {\mathbf v}_i \Vert = 1$ for $i=1,2,\ldots,n.$

If the set $\{{\mathbf v}_1, {\mathbf v}_2, \ldots, {\mathbf v}_n \}$ is also a basis of $V,$ then the set of vectors $\{{\mathbf v}_1, {\mathbf v}_2, \ldots, {\mathbf v}_n \}$ is called an orthonormal basis of $V.$

EXAMPLE 5.1.14  

1. Consider the vector space ${\mathbb{R}}^2$ with the standard inner product. Then the standard ordered basis ${\cal B}= \bigl( (1,0), (0,1) \bigr)$ is an orthonormal set. Also, the basis ${\cal B}_1 = \displaystyle \bigl( \frac{1}{\sqrt{2}}(1,1), \frac{1}{\sqrt{2}}(1,-1) \bigr)$ is an orthonormal set.
2. Let ${\mathbb{R}}^n$ be endowed with the standard inner product. Then by Exercise 5.1.10.1, the standard ordered basis $({\mathbf e}_1, {\mathbf e}_2, \ldots, {\mathbf e}_n)$ is an orthonormal set.

In view of Theorem 5.1.12, we inquire into the question of extracting an orthonormal basis from a given basis. In the next section, we describe a process (called the Gram-Schmidt Orthogonalisation process) that generates an orthonormal set from a given set containing finitely many vectors.

Remark 5.1.15   The last part of the above theorem can be rephrased as ``suppose $\{{\mathbf v}_1, {\mathbf v}_2, \ldots, {\mathbf v}_n \}$ is an orthonormal basis of an inner product space $V.$ Then for each ${\mathbf u}\in V$ the numbers $\langle {\mathbf u}, {\mathbf v}_i \rangle$ for $1 \leq i \leq n$ are the coordinates of ${\mathbf u}$ with respect to the above basis".

That is, let ${\cal B}= ({\mathbf v}_1, {\mathbf v}_2, \ldots, {\mathbf v}_n)$ be an ordered basis. Then for any ${\mathbf u}\in V,$

$$[{\mathbf u}]_{{\cal B}} = ( \langle {\mathbf u}, {\mathbf v}_1\r... ..., {\mathbf v}_2 \rangle, \ldots, \langle {\mathbf u}, {\mathbf v}_n \rangle)^t.$$