1.7 Linear Independence

Theory: Linear Independence

Linear Independence (Lay §1.7)

An indexed set of vectors $\{\mathbf{v}_1, \dots, \mathbf{v}_p\}$ in $\mathbb{R}^n$ is linearly independent if the vector equation

$$x_1\mathbf{v}_1 + x_2\mathbf{v}_2 + \cdots + x_p\mathbf{v}_p = \mathbf{0}$$

has only the trivial solution $x_1 = \cdots = x_p = 0$. The set is linearly dependent if there are weights $c_1, \dots, c_p$, not all zero, with $c_1\mathbf{v}_1 + \cdots + c_p\mathbf{v}_p = \mathbf{0}$. Such an equation is called a linear dependence relation.

Written as a matrix equation, the vector equation is just a homogeneous system:

$$\begin{bmatrix} \mathbf{v}_1 & \cdots & \mathbf{v}_p \end{bmatrix}\begin{bmatrix} c_1 \\ \vdots \\ c_p \end{bmatrix} = c_1\mathbf{v}_1 + \cdots + c_p\mathbf{v}_p = \mathbf{0}$$

• only the trivial solution (every $c = 0$) $\Rightarrow$ independent
• at least one $c$ nonzero possible $\Rightarrow$ dependent

Linear independence of matrix columns (Lay §1.7): The columns of a matrix $A$ are linearly independent if and only if the equation $A\mathbf{x} = \mathbf{0}$ has only the trivial solution — i.e. iff $A$ has a pivot in every column (no free variables).

So the standard test is: put the vectors as columns, row reduce, and look for a free variable.

Mini-example

Are $\mathbf{v}_1 = (1, 2, 3)$, $\mathbf{v}_2 = (4, 5, 6)$, $\mathbf{v}_3 = (2, 1, 0)$ independent?

$$\begin{bmatrix} 1 & 4 & 2 \\ 2 & 5 & 1 \\ 3 & 6 & 0 \end{bmatrix} \sim \begin{bmatrix} 1 & 4 & 2 \\ 0 & -3 & -3 \\ 0 & -6 & -6 \end{bmatrix} \sim \begin{bmatrix} 1 & 4 & 2 \\ 0 & -3 & -3 \\ 0 & 0 & 0 \end{bmatrix}$$

($R_2 \leftarrow R_2 - 2R_1$, $R_3 \leftarrow R_3 - 3R_1$, then $R_3 \leftarrow R_3 - 2R_2$.) Column 3 has no pivot, so $x_3$ is free → nontrivial solutions exist → dependent. (In fact $\mathbf{v}_3 = -2\mathbf{v}_1 + \mathbf{v}_2$.)

Quick Tests (no row reduction needed)

These are the “by inspection” rules — learn them, they save a lot of time in exams.

Situation	Conclusion	Reason
One vector $\{\mathbf{v}\}$	independent iff $\mathbf{v} \ne \mathbf{0}$	$x\mathbf{v} = \mathbf{0}$ forces $x = 0$ when $\mathbf{v} \ne \mathbf{0}$
Two vectors $\{\mathbf{v}_1, \mathbf{v}_2\}$	dependent iff one is a multiple of the other (e.g. $\mathbf{v}_1 = 3\mathbf{v}_2$)	geometrically: both lie on one line through the origin
Set contains $\mathbf{0}$	dependent (Theorem 9)	weight 1 on $\mathbf{0}$, weight 0 on the rest
More vectors than entries ($p > n$)	dependent (Theorem 8)	more columns than rows → not every column can have a pivot

Theorem 7 — Characterization of Linearly Dependent Sets (Lay §1.7): An indexed set $S = \{\mathbf{v}_1, \dots, \mathbf{v}_p\}$ of two or more vectors is linearly dependent if and only if at least one of the vectors in $S$ is a linear combination of the others. In fact, if $S$ is dependent and $\mathbf{v}_1 \ne \mathbf{0}$, then some $\mathbf{v}_j$ (with $j > 1$) is a linear combination of the preceding vectors $\mathbf{v}_1, \dots, \mathbf{v}_{j-1}$.

Common mistake: Theorem 7 does not say that every vector in a dependent set is a combination of the others. E.g. in $\{(1,0), (2,0), (0,1)\}$, the set is dependent ($\mathbf{v}_2 = 2\mathbf{v}_1$), but $(0,1)$ is not a combination of the other two.

Theorem 8 (Lay §1.7): If a set contains more vectors than there are entries in each vector, then the set is linearly dependent. That is, any set $\{\mathbf{v}_1, \dots, \mathbf{v}_p\}$ in $\mathbb{R}^n$ is dependent if $p > n$.

Theorem 9 (Lay §1.7): If a set $S = \{\mathbf{v}_1, \dots, \mathbf{v}_p\}$ in $\mathbb{R}^n$ contains the zero vector, then the set is linearly dependent.

Why Theorem 8 works: the matrix $A = [\mathbf{v}_1 \cdots \mathbf{v}_p]$ is $n \times p$. There are at most $n$ pivots (one per row), but $p > n$ columns, so at least one column has no pivot → free variable → nontrivial solution of $A\mathbf{x} = \mathbf{0}$.

Example: three vectors in $\mathbb{R}^2$ ($p = 3$ vectors, $n = 2$ entries each), such as $(1,2), (3,4), (5,6)$, are automatically dependent.

Theorem 8 only works in one direction. If $p \le n$, you learn nothing — the set may be dependent or independent (e.g. $(1,1,1)$ and $(2,2,2)$ are only 2 vectors in $\mathbb{R}^3$ but are dependent). You must then row reduce or use the other tests.

---

Solved Examples (Textbook Questions)

Exercise 1

(Adapted from Lay §1.7, Exercises 9 & 10)

For each set, (a) for what values of $h$ is $\mathbf{v}_3$ in $\operatorname{Span}\{\mathbf{v}_1, \mathbf{v}_2\}$, and (b) for what values of $h$ is $\{\mathbf{v}_1, \mathbf{v}_2, \mathbf{v}_3\}$ linearly dependent?

Ex. 9: $\mathbf{v}_1 = \begin{bmatrix} 1 \\ -3 \\ 2 \end{bmatrix}, \ \mathbf{v}_2 = \begin{bmatrix} -3 \\ 9 \\ -6 \end{bmatrix}, \ \mathbf{v}_3 = \begin{bmatrix} 5 \\ -7 \\ h \end{bmatrix}$ $\qquad$ Ex. 10: $\mathbf{v}_1 = \begin{bmatrix} 1 \\ -5 \\ -3 \end{bmatrix}, \ \mathbf{v}_2 = \begin{bmatrix} -2 \\ 10 \\ 6 \end{bmatrix}, \ \mathbf{v}_3 = \begin{bmatrix} 2 \\ -9 \\ h \end{bmatrix}$

Show solution ↓Hide solution ↑

Solution

Exercise 9.

(a) $\mathbf{v}_3 \in \operatorname{Span}\{\mathbf{v}_1, \mathbf{v}_2\}$ iff $x_1\mathbf{v}_1 + x_2\mathbf{v}_2 = \mathbf{v}_3$ is consistent. Row reduce $[\mathbf{v}_1 \ \mathbf{v}_2 \mid \mathbf{v}_3]$ with $R_2 \leftarrow R_2 + 3R_1$, $R_3 \leftarrow R_3 - 2R_1$:

$$\left[\begin{array}{cc|c} 1 & -3 & 5 \\ -3 & 9 & -7 \\ 2 & -6 & h \end{array}\right] \sim \left[\begin{array}{cc|c} 1 & -3 & 5 \\ 0 & 0 & 8 \\ 0 & 0 & h - 10 \end{array}\right]$$

Row 2 reads $0 = 8$: the augmented column has a pivot, so the system is inconsistent for every $h$. $\mathbf{v}_3$ is in the span for no value of $h$.

(b) Row reduce $[\mathbf{v}_1 \ \mathbf{v}_2 \ \mathbf{v}_3 \mid \mathbf{0}]$ — same steps, then $R_3 \leftarrow R_3 - \tfrac{h-10}{8}R_2$:

$$\left[\begin{array}{ccc|c} 1 & -3 & 5 & 0 \\ 0 & 0 & 8 & 0 \\ 0 & 0 & h - 10 & 0 \end{array}\right] \sim \left[\begin{array}{ccc|c} 1 & -3 & 5 & 0 \\ 0 & 0 & 8 & 0 \\ 0 & 0 & 0 & 0 \end{array}\right]$$

Column 2 never has a pivot, so $x_2$ is free for every $h$ → nontrivial solution → the set is dependent for all $h$. (By inspection: $\mathbf{v}_2 = -3\mathbf{v}_1$, so $\{\mathbf{v}_1, \mathbf{v}_2\}$ is already dependent.)

Exercise 10. Same pattern ($\mathbf{v}_2 = -2\mathbf{v}_1$). With $R_2 \leftarrow R_2 + 5R_1$, $R_3 \leftarrow R_3 + 3R_1$:

$$\left[\begin{array}{cc|c} 1 & -2 & 2 \\ -5 & 10 & -9 \\ -3 & 6 & h \end{array}\right] \sim \left[\begin{array}{cc|c} 1 & -2 & 2 \\ 0 & 0 & 1 \\ 0 & 0 & h + 6 \end{array}\right]$$

(a) Row 2 says $0 = 1$ → inconsistent → $\mathbf{v}_3 \in \operatorname{Span}\{\mathbf{v}_1, \mathbf{v}_2\}$ for no $h$.

(b) For $[\mathbf{v}_1 \ \mathbf{v}_2 \ \mathbf{v}_3 \mid \mathbf{0}]$, after $R_3 \leftarrow R_3 - (h+6)R_2$ the pivots are in columns 1 and 3 only, so $x_2$ is free → dependent for all $h$.

Parts (a) and (b) have different answers! (a) asks about $\mathbf{v}_3$ specifically; (b) asks whether any vector is a combination of the others (Theorem 7) — here $\mathbf{v}_2$ is.

Exercise 2

(Adapted from Lay §1.7, Exercises 11–14)

Find the value(s) of $h$ for which the vectors are linearly dependent.

11. $\begin{bmatrix} 1 \\ -1 \\ 4 \end{bmatrix}, \begin{bmatrix} 3 \\ -5 \\ 7 \end{bmatrix}, \begin{bmatrix} -1 \\ 5 \\ h \end{bmatrix}$ $\qquad$ 12. $\begin{bmatrix} 2 \\ -4 \\ 1 \end{bmatrix}, \begin{bmatrix} -6 \\ 7 \\ -3 \end{bmatrix}, \begin{bmatrix} 8 \\ h \\ 4 \end{bmatrix}$

13. $\begin{bmatrix} 1 \\ 5 \\ -3 \end{bmatrix}, \begin{bmatrix} -2 \\ -9 \\ 6 \end{bmatrix}, \begin{bmatrix} 3 \\ h \\ -9 \end{bmatrix}$ $\qquad$ 14. $\begin{bmatrix} 1 \\ -1 \\ 3 \end{bmatrix}, \begin{bmatrix} -5 \\ 7 \\ 8 \end{bmatrix}, \begin{bmatrix} 1 \\ 1 \\ h \end{bmatrix}$

Show solution ↓Hide solution ↑

Solution

Method: put the vectors as columns of $A$ and row reduce. The set is dependent exactly when some column has no pivot (a free variable in $A\mathbf{x} = \mathbf{0}$). The zero augmented column never changes, so we omit it.

11. $R_2 \leftarrow R_2 + R_1$, $R_3 \leftarrow R_3 - 4R_1$, then $R_3 \leftarrow R_3 - \tfrac{5}{2}R_2$:

$$\begin{bmatrix} 1 & 3 & -1 \\ -1 & -5 & 5 \\ 4 & 7 & h \end{bmatrix} \sim \begin{bmatrix} 1 & 3 & -1 \\ 0 & -2 & 4 \\ 0 & -5 & h + 4 \end{bmatrix} \sim \begin{bmatrix} 1 & 3 & -1 \\ 0 & -2 & 4 \\ 0 & 0 & h - 6 \end{bmatrix}$$

$x_3$ is free iff $h - 6 = 0$. Dependent iff $h = 6$.

12. $R_2 \leftarrow R_2 + 2R_1$, $R_3 \leftarrow R_3 - \tfrac12 R_1$:

$$\begin{bmatrix} 2 & -6 & 8 \\ -4 & 7 & h \\ 1 & -3 & 4 \end{bmatrix} \sim \begin{bmatrix} 2 & -6 & 8 \\ 0 & -5 & h + 16 \\ 0 & 0 & 0 \end{bmatrix}$$

Only 2 pivots for 3 columns, regardless of $h$ → $x_3$ always free → dependent for all $h$. (Notice $\mathbf{v}_3$‘s first and third entries are $4\times$ those of $\mathbf{v}_1$; row 3 of $A$ is $\tfrac12$ row 1.)

13. $R_2 \leftarrow R_2 - 5R_1$, $R_3 \leftarrow R_3 + 3R_1$:

$$\begin{bmatrix} 1 & -2 & 3 \\ 5 & -9 & h \\ -3 & 6 & -9 \end{bmatrix} \sim \begin{bmatrix} 1 & -2 & 3 \\ 0 & 1 & h - 15 \\ 0 & 0 & 0 \end{bmatrix}$$

Again always a free variable → dependent for all $h$.

14. $R_2 \leftarrow R_2 + R_1$, $R_3 \leftarrow R_3 - 3R_1$, then $R_3 \leftarrow R_3 - \tfrac{23}{2}R_2$:

$$\begin{bmatrix} 1 & -5 & 1 \\ -1 & 7 & 1 \\ 3 & 8 & h \end{bmatrix} \sim \begin{bmatrix} 1 & -5 & 1 \\ 0 & 2 & 2 \\ 0 & 23 & h - 3 \end{bmatrix} \sim \begin{bmatrix} 1 & -5 & 1 \\ 0 & 2 & 2 \\ 0 & 0 & h - 26 \end{bmatrix}$$

Dependent iff $h = 26$. Check: with $h = 26$, $\mathbf{v}_3 = 6\mathbf{v}_1 + \mathbf{v}_2 = (6 - 5,\ -6 + 7,\ 18 + 8) = (1, 1, 26)$ ✓.

Exercise 3

(Adapted from Lay §1.7, Exercises 15–20)

Determine by inspection whether the vectors are linearly independent. Justify each answer.

15. $\begin{bmatrix} 5 \\ 1 \end{bmatrix}, \begin{bmatrix} 2 \\ 8 \end{bmatrix}, \begin{bmatrix} 1 \\ 3 \end{bmatrix}, \begin{bmatrix} -1 \\ 7 \end{bmatrix}$ $\qquad$ 16. $\begin{bmatrix} 4 \\ -2 \\ 6 \end{bmatrix}, \begin{bmatrix} 6 \\ -3 \\ 9 \end{bmatrix}$ $\qquad$ 17. $\begin{bmatrix} 3 \\ 5 \\ -1 \end{bmatrix}, \begin{bmatrix} 0 \\ 0 \\ 0 \end{bmatrix}, \begin{bmatrix} -6 \\ 5 \\ 4 \end{bmatrix}$

18. $\begin{bmatrix} 4 \\ 4 \end{bmatrix}, \begin{bmatrix} -1 \\ 3 \end{bmatrix}, \begin{bmatrix} 2 \\ 5 \end{bmatrix}, \begin{bmatrix} 8 \\ 1 \end{bmatrix}$ $\qquad$ 19. $\begin{bmatrix} -8 \\ 12 \\ -4 \end{bmatrix}, \begin{bmatrix} 2 \\ -3 \\ -1 \end{bmatrix}$ $\qquad$ 20. $\begin{bmatrix} 1 \\ 4 \\ -7 \end{bmatrix}, \begin{bmatrix} -2 \\ 5 \\ 3 \end{bmatrix}, \begin{bmatrix} 0 \\ 0 \\ 0 \end{bmatrix}$

Show solution ↓Hide solution ↑

Solution

Run through the quick tests in order: zero vector? more vectors than entries? two vectors — multiples?

Ex.	Test that applies	Verdict
15	4 vectors with only 2 entries each ($p = 4 > n = 2$), Theorem 8	dependent
16	two vectors: $(6, -3, 9) = \tfrac32 (4, -2, 6)$	dependent
17	contains the zero vector, Theorem 9	dependent
18	4 vectors in $\mathbb{R}^2$, Theorem 8	dependent
19	two vectors, not multiples: first entries give factor $-8/2 = -4$, and $-4\cdot(-3) = 12$ ✓, but $-4\cdot(-1) = 4 \ne -4$ ✗	independent
20	contains the zero vector, Theorem 9	dependent

In 15 and 18 you don’t need to find the actual dependence relation — Theorem 8 guarantees one exists. In 19, one mismatching entry is enough to kill “multiple”.