System of Linear Equations

THEOREM 15.1.1 (Existence and Non-existence)   Consider a linear system $A {\mathbf x}= {\mathbf b},$ where $A$ is a $m \times n$ matrix, and $\; {\mathbf x}, \; {\mathbf b}$ are vectors with orders $n \times 1,$ and $m \times 1,$ respectively. Suppose ${\mbox{rank }}(A) = r$ and ${\mbox{rank}} ([A \; \;{\mathbf b}]) = r_a.$ Then exactly one of the following statement holds:

1. if $\; r_a = r < n,$ the set of solutions of the linear system is an infinite set and has the form
   $$\{ {\mathbf u}_0 + k_1 {\mathbf u}_1 + k_2 {\mathbf u}_2 + \cdots... ...n-r} {\mathbf u}_{n-r} \; : \;\; k_i \in {\mathbb{R}}, \; 1 \leq i \leq n-r \},$$
   where ${\mathbf u}_0, {\mathbf u}_1, \ldots, {\mathbf u}_{n-r}$ are $n \times 1$ vectors satisfying $A {\mathbf u}_0 = {\mathbf b}$ and $A {\mathbf u}_i = {\mathbf 0}$ for $1 \leq i \leq n-r.$
2. if $\; r_a = r = n,$ the solution set of the linear system has a unique $n \times 1$ vector ${\mathbf x}_0$ satisfying $A {\mathbf x}_0 = {\mathbf 0}.$
3. If $\; r < r_a,$ the linear system has no solution.

Proof. Suppose $[C \; \; {\mathbf d}]$ is the row reduced echelon form of the augmented matrix $[ A \; \; {\mathbf b}].$ Then by Theorem 2.2.5, the solution set of the linear system $[C \; \; {\mathbf d}]$ is same as the solution set of the linear system $[ A \; \; {\mathbf b}].$ So, the proof consists of understanding the solution set of the linear system $C {\mathbf x}= {\mathbf d}.$

1. Let $r = r_a < n.$

   Then $[C \; \; {\mathbf d}]$ has its first $r$ rows as the non-zero rows. So, by Remark 2.3.5, the matrix $C = [c_{ij}]$ has $r$ leading columns. Let the leading columns be $1 \leq i_1 < i_2 < \cdots < i_r \leq n.$ Then we observe the following:

   1. the entries $c_{l i_l}$ for $1 \leq l \leq r$ are leading terms. That is, for $1 \leq l \leq r,$ all entries in the $i_l^{\mbox{th}}$ column of $C$ is zero, except the entry $c_{l i_l}.$ The entry $c_{l i_l}=1;$
   2. corresponding is each leading column, we have $r$ BASIC VARIABLES, $x_{i_1}, x_{i_2}, \ldots, x_{i_r};$
   3. the remaining $n-r$ columns correspond to the $n-r$ FREE VARIABLES (see Remark 2.3.5), $x_{j_1}, x_{j_2}, \ldots, x_{j_{n-r}}.$ So, the free variables correspond to the columns $1\leq j_1 < j_2 < \cdots < j_{n-r}\leq n.$
   For $1 \leq l \leq r,$ consider the $l^{\mbox{th}}$ row of $[C \;\; {\mathbf d}].$ The entry $c_{l i_{l}} = 1$ and is the leading term. Also, the first $r$ rows of the augmented matrix $[C \; \; {\mathbf d}]$ give rise to the linear equations
   $$x_{i_l} + \sum\limits_{k=1}^{n-r} c_{{l} j_k} x_{j_k}= d_l, \;\; {\mbox{ for }} \;\; 1 \leq l \leq r.$$
   These equations can be rewritten as
   $$x_{i_l} = d_l - \sum\limits_{k=1}^{n-r} c_{{l} j_k} x_{j_k}= d_l, \;\; {\mbox{ for }} \;\; 1 \leq l \leq r.$$
   Let ${\mathbf y}^t = (x_{i_1}, \ldots, x_{i_r}, x_{j_1}, \ldots, x_{j_{n-r}}).$ Then the set of solutions consists of

   $${\mathbf y}= \begin{bmatrix}x_{i_1} \\ \vdots \\ x_{i_r} \\ x_{j_... ...1}^{n-r} c_{{r} j_k} x_{j_k} \\ x_{j_1} \\ \vdots \\ x_{j_{n-r}} \end{bmatrix}.$$ (15.1.1)

   
   
   As $x_{j_s}$ for $1 \leq s \leq n-r$ are free variables, let us assign arbitrary constants $k_s \in {\mathbb{R}}$ to $x_{j_s}.$ That is, for $1 \leq s \leq n-r, \;\; x_{j_s} = k_s.$ Then the set of solutions is given by
   

   $${\mathbf y} = \begin{bmatrix} d_{1} - \sum\limits_{s=1}^{n-r} c_{{1} j_s} x_{j_... ...m\limits_{s=1}^{n-r} c_{{r} j_s} k_s \\ k_1 \\ \vdots \\ k_{n-r} \end{bmatrix}$$

   $$= \begin{bmatrix}d_{1} \\ \vdots \\ d_{r} \\ 0 \\ 0 \\ \vdots \\ 0 ... ...r}} \\ \vdots \\ c_{{r} j_{n-r}} \\ 0 \\ 0 \\ \vdots \\ 0 \\ - 1 \end{bmatrix}.$$

   
   Let us write ${{\mathbf v}_0}^t = (d_1, d_2, \ldots, d_r, 0, \ldots, 0)^t.$ Also, for $1 \leq i \leq n-r,$ let ${\mathbf v}_i$ be the vector associated with $k_i$ in the above representation of the solution ${\mathbf y}.$ Observe the following:
   1. if we assign $k_s = 0, \;$ for $1 \leq s \leq n-r,$ we get

      $$C {\mathbf v}_0 = C {\mathbf y}= {\mathbf d}.$$ (15.1.2)

      
      

   2. if we assign $k_1 = 1$ and $k_s = 0, \;$ for $2 \leq s \leq n-r,$ we get

      $${\mathbf d}= C {\mathbf y}= C ({\mathbf v}_0+ {\mathbf v}_1).$$ (15.1.3)

      
      
      So, using (14.1.2), we get $C {\mathbf v}_1 = {\mathbf 0}.$
   3. in general, if we assign $k_t = 1$ and $k_s = 0, \;$ for $1 \leq s \neq t \leq n-r,$ we get

      $${\mathbf d}= C {\mathbf y}= C ({\mathbf v}_0+ {\mathbf v}_t).$$ (15.1.4)

      
      
      So, using (14.1.2), we get $C {\mathbf v}_t = {\mathbf 0}.$
   Note that a rearrangement of the entries of ${\mathbf y}$ will give us the solution vector ${\mathbf x}^t = (x_1, x_2, \ldots, x_n)^t.$ Suppose that for $0 \leq i \leq n-r,$ the vectors ${\mathbf u}_i$ 's are obtained by applying the same rearrangement to the entries of ${\mathbf v}_i$ 's which when applied to ${\mathbf y}$ gave ${\mathbf x}.$ Therefore, we have $C {\mathbf u}_0 = {\mathbf d}$ and for $1 \leq i \leq n-r,$ $C {\mathbf u}_i = {\mathbf 0}.$ Now, using equivalence of the linear system $A {\mathbf x}= {\mathbf b}$ and $C {\mathbf x}= {\mathbf d}$ gives
   $$A {\mathbf u}_0 = {\mathbf b}\;\; {\mbox{ and for }} \;\; 1 \leq i \leq n-r, \; A {\mathbf u}_i = {\mathbf 0}.$$
   Thus, we have obtained the desired result for the case $r = r_1 < n.$
2. $r= r_a = n, \; m \geq n.$

   Here the first $n$ rows of the row reduced echelon matrix $[C \; \; {\mathbf d}]$ are the non-zero rows. Also, the number of columns in $C$ equals $n = {\mbox{ rank }}(A) = {\mbox{ rank }}(C).$ So, by Remark 2.3.5, all the columns of $C$ are leading columns and all the variables $x_1, x_2, \ldots, x_n$ are basic variables. Thus, the row reduced echelon form $[C \; \; {\mathbf d}]$ of $[A \;\; {\mathbf b}]$ is given by

   $$[C \;\; {\mathbf d}] = \begin{bmatrix}I_n & \tilde{{\mathbf d}}\\ {\mathbf 0}& {\mathbf 0}\end{bmatrix}.$$
   Therefore, the solution set of the linear system $C {\mathbf x}= {\mathbf d}$ is obtained using the equation $I_n {\mathbf x}= \tilde{{\mathbf d}}.$ This gives us, a solution as ${\mathbf x}_0 = \tilde{{\mathbf d}}.$ Also, by Theorem 2.3.11, the row reduced form of a given matrix is unique, the solution obtained above is the only solution. That is, the solution set consists of a single vector $\tilde{{\mathbf d}}.$
3. $r < r_a.$

   As $C$ has $n$ columns, the row reduced echelon matrix $[C \; \; {\mathbf d}]$ has $n+1$ columns. The condition, $r < r_a$ implies that $r_a = r+1.$ We now observe the following:

   1. as ${\mbox{rank}} (C) = r,$ the $(r+1)$ th row of $C$ consists of only zeros.
   2. Whereas the condition $r_a = r+1$ implies that the $(r+1)^{\mbox{th}}$ row of the matrix $[C \; \; {\mathbf d}]$ is non-zero.
   Thus, the $(r+1)^{\mbox{th}}$ row of $[C \; \; {\mathbf d}]$ is of the form $(0,\ldots, 0, 1).$ Or in other words, ${\mathbf d}_{r+1} = 1.$

   Thus, for the equivalent linear system $C {\mathbf x}= {\mathbf d},$ the $(r+1)^{\mbox{th}}$ equation is

   $$0 \; x_1 + 0 \; x_2 + \cdots + 0 \; x_n = 1.$$
   This linear equation has no solution. Hence, in this case, the linear system $C {\mathbf x}= {\mathbf d}$ has no solution. Therefore, by Theorem 2.2.5, the linear system $A {\mathbf x}= {\mathbf b}$ has no solution.

height6pt width 6pt depth 0pt

We now state a corollary whose proof is immediate from previous results.

COROLLARY 15.1.2   Consider the linear system $A {\mathbf x}= {\mathbf b}.$ Then the two statements given below cannot hold together.

1. The system $A {\mathbf x}= {\mathbf b}$ has a unique solution for every ${\mathbf b}.$
2. The system $A {\mathbf x}= {\mathbf 0}$ has a non-trivial solution.