2.1 Matrix Operations

Theory: Matrix Operations

Matrix Notation (Lay §2.1)

An $m \times n$ matrix $A$ has $m$ rows and $n$ columns. We write it by columns or by entries:

$$A = \begin{bmatrix} \mathbf{a}_1 & \mathbf{a}_2 & \cdots & \mathbf{a}_n \end{bmatrix} = \begin{bmatrix} a_{11} & \cdots & a_{1n} \\ \vdots & & \vdots \\ a_{m1} & \cdots & a_{mn} \end{bmatrix}$$

• $a_{ij}$ is the entry in row $i$, column $j$ (row first, always).
• The diagonal entries are $a_{11}, a_{22}, a_{33}, \dots$; a diagonal matrix is square with zeros everywhere off the diagonal (e.g. the identity $I_n$).
• The zero matrix $0$ has every entry $0$.
• Two matrices are equal when they have the same size and the same corresponding entries.

Sums and Scalar Multiples

• $A + B$ is defined only when $A$ and $B$ have the same size: add entry by entry (column by column).
• $rA$ multiplies every entry of $A$ by the scalar $r$. We write $-A$ for $(-1)A$ and $A - B$ for $A + (-1)B$.

Theorem 1 (Lay §2.1): Let $A$, $B$, $C$ be matrices of the same size and $r$, $s$ scalars. Then

1. $A + B = B + A$
2. $(A + B) + C = A + (B + C)$
3. $A + 0 = A$
4. $r(A + B) = rA + rB$
5. $(r + s)A = rA + sA$
6. $r(sA) = (rs)A$

In short: matrix addition and scalar multiplication behave exactly like ordinary vector algebra.

Matrix Multiplication

Multiplying by $B$ and then by $A$ (that is, $\mathbf{x} \mapsto B\mathbf{x} \mapsto A(B\mathbf{x})$) should be a single matrix $AB$. That is what motivates the definition.

Definition (Lay §2.1): If $A$ is $m \times n$ and $B = \begin{bmatrix} \mathbf{b}_1 & \mathbf{b}_2 & \cdots & \mathbf{b}_p \end{bmatrix}$ is $n \times p$, then

$$AB = A\begin{bmatrix} \mathbf{b}_1 & \mathbf{b}_2 & \cdots & \mathbf{b}_p \end{bmatrix} = \begin{bmatrix} A\mathbf{b}_1 & A\mathbf{b}_2 & \cdots & A\mathbf{b}_p \end{bmatrix}$$

Each column of $AB$ is a linear combination of the columns of $A$, using weights from the corresponding column of $B$.

The Size Rule

$$\underset{m \times n}{A} \;\; \underset{n \times p}{B} = \underset{m \times p}{AB}$$

The inner numbers must match ($n = n$); the outer numbers give the size of the product.

Example. If $A$ is $3 \times 2$ and $B$ is $2 \times 4$, then $AB$ is $3 \times 4$, but $BA$ is undefined ($B$ has 4 columns, $A$ has 3 rows).

The Row–Column Rule

For hand computation, compute one entry at a time: entry $(i, j)$ of $AB$ is (row $i$ of $A$) $\cdot$ (column $j$ of $B$):

$$(AB)_{ij} = a_{i1}b_{1j} + a_{i2}b_{2j} + \cdots + a_{in}b_{nj}$$

Mini-example. With $A = \begin{bmatrix} 2 & 3 \\ 1 & -5 \end{bmatrix}$ and $B = \begin{bmatrix} 4 & 3 & 6 \\ 1 & -2 & 3 \end{bmatrix}$, entry $(1,3)$ of $AB$ is row 1 of $A$ times column 3 of $B$: $2(6) + 3(3) = 21$.

Similarly, a whole row of $AB$ is (row $i$ of $A$) times $B$:

$$\operatorname{row}_i(AB) = \operatorname{row}_i(A) \cdot B$$

Example 6 (Lay §2.1). Find the second row of $AB$, where

$$A = \begin{bmatrix} 2 & -5 & 0 \\ -1 & 3 & -4 \\ 6 & -8 & -7 \\ -3 & 0 & 9 \end{bmatrix}, \qquad B = \begin{bmatrix} 4 & -6 \\ 7 & 1 \\ 3 & 2 \end{bmatrix}$$

Only row 2 of $A$ is needed:

$$\begin{bmatrix} -1 & 3 & -4 \end{bmatrix}\begin{bmatrix} 4 & -6 \\ 7 & 1 \\ 3 & 2 \end{bmatrix} = \begin{bmatrix} -4 + 21 - 12 & 6 + 3 - 8 \end{bmatrix} = \begin{bmatrix} 5 & 1 \end{bmatrix}$$

How to recognise the question type: “Compute $AB$ in two ways” means (a) by the definition, computing $A\mathbf{b}_1, A\mathbf{b}_2, \dots$ separately and placing them side by side, and (b) by the row–column rule. “Find the second row of $AB$” means: you only need row 2 of $A$ (times all of $B$). “Find the first column of $B$ given $A$ and $AB$” means: solve $A\mathbf{x} = \text{(first column of } AB)$.

Properties of Matrix Multiplication

Theorem 2 (Lay §2.1): Let $A$ be $m \times n$, and let $B$, $C$ have sizes for which the sums and products are defined. Then

1. $A(BC) = (AB)C$ (associative law)
2. $A(B + C) = AB + AC$ (left distributive law)
3. $(B + C)A = BA + CA$ (right distributive law)
4. $r(AB) = (rA)B = A(rB)$ for any scalar $r$
5. $I_m A = A = A I_n$ (identity for multiplication)

So $IA = AI = A$: the identity matrix plays the role of the number $1$.

Common mistakes — matrix algebra is NOT number algebra:

1. In general $AB \ne BA$. Order matters: ”$A$ times $B$ on the right” ($AB$) differs from ”$B$ times $A$” ($BA$). Even when both are defined and the same size, they are usually different.
2. No cancellation: $AB = AC$ does not imply $B = C$ (see Exercise 3 below, textbook Exercise 10).
3. $AB = 0$ does not imply $A = 0$ or $B = 0$. For instance $\begin{bmatrix} 1 & 1 \\ 1 & 1 \end{bmatrix}\begin{bmatrix} 1 & -1 \\ -1 & 1 \end{bmatrix} = \begin{bmatrix} 0 & 0 \\ 0 & 0 \end{bmatrix}$.

Powers of a Matrix

If $A$ is square ($n \times n$) and $k$ is a positive integer,

$$A^k = \underbrace{A \cdots A}_{k \text{ factors}}, \qquad A^0 = I_n$$

$A^k \mathbf{x}$ means applying the transformation $\mathbf{x} \mapsto A\mathbf{x}$ $k$ times. Note $(A + B)^2 = A^2 + AB + BA + B^2$, not $A^2 + 2AB + B^2$ (since $AB \ne BA$ in general).

The Transpose of a Matrix

The transpose $A^T$ of an $m \times n$ matrix $A$ is the $n \times m$ matrix whose columns are the rows of $A$ (equivalently, $(A^T)_{ij} = a_{ji}$).

Example 8 (Lay §2.1).

$$A = \begin{bmatrix} a & b \\ c & d \end{bmatrix}, \quad B = \begin{bmatrix} -5 & 2 \\ 1 & -3 \\ 0 & 4 \end{bmatrix}, \quad C = \begin{bmatrix} 1 & 1 & 1 & 1 \\ -3 & 5 & -2 & 7 \end{bmatrix}$$ $$A^T = \begin{bmatrix} a & c \\ b & d \end{bmatrix}, \quad B^T = \begin{bmatrix} -5 & 1 & 0 \\ 2 & -3 & 4 \end{bmatrix}, \quad C^T = \begin{bmatrix} 1 & -3 \\ 1 & 5 \\ 1 & -2 \\ 1 & 7 \end{bmatrix}$$

Theorem 3 (Lay §2.1): Let $A$ and $B$ be matrices whose sizes are appropriate for the following sums and products.

1. $(A^T)^T = A$
2. $(A + B)^T = A^T + B^T$
3. $(rA)^T = rA^T$ for any scalar $r$
4. $(AB)^T = B^T A^T$

The transpose of a product is the product of the transposes in reverse order. (Check the sizes: if $A$ is $m \times n$ and $B$ is $n \times p$, then $B^T A^T$ is $(p \times n)(n \times m)$, which works; $A^T B^T$ usually does not even exist.)

---

Solved Examples (Textbook Questions)

Exercise 1

(Adapted from Lay §2.1, Exercise 3)

Let $A = \begin{bmatrix} 4 & -1 \\ 5 & -2 \end{bmatrix}$. Compute $3I_2 - A$ and $(3I_2)A$.

Show solution ↓Hide solution ↑

Solution

$3I_2 - A$: $3I_2$ has $3$ on the diagonal, so only the diagonal entries of $A$ are subtracted from $3$; off-diagonal entries just change sign.

$$3I_2 - A = \begin{bmatrix} 3 & 0 \\ 0 & 3 \end{bmatrix} - \begin{bmatrix} 4 & -1 \\ 5 & -2 \end{bmatrix} = \begin{bmatrix} 3 - 4 & 0 - (-1) \\ 0 - 5 & 3 - (-2) \end{bmatrix} = \begin{bmatrix} -1 & 1 \\ -5 & 5 \end{bmatrix}$$

$(3I_2)A$: by Theorem 2, $(3I_2)A = 3(I_2A) = 3A$:

$$(3I_2)A = 3\begin{bmatrix} 4 & -1 \\ 5 & -2 \end{bmatrix} = \begin{bmatrix} 12 & -3 \\ 15 & -6 \end{bmatrix}$$

Check with the row–column rule:

$$\begin{bmatrix} 3 & 0 \\ 0 & 3 \end{bmatrix}\begin{bmatrix} 4 & -1 \\ 5 & -2 \end{bmatrix} = \begin{bmatrix} 3 \cdot 4 + 0 & 3(-1) + 0 \\ 0 + 3 \cdot 5 & 0 + 3(-2) \end{bmatrix} = \begin{bmatrix} 12 & -3 \\ 15 & -6 \end{bmatrix} \checkmark$$

Exercise 2

(Adapted from Lay §2.1, Exercises 5 and 6)

Compute the product $AB$ in two ways: (a) by the definition, where $A\mathbf{b}_1$ and $A\mathbf{b}_2$ are computed separately, and (b) by the row–column rule.

1. $A = \begin{bmatrix} -1 & 2 \\ 5 & 4 \\ 2 & -3 \end{bmatrix}$, $B = \begin{bmatrix} 3 & -2 \\ -2 & 1 \end{bmatrix}$
2. $A = \begin{bmatrix} 4 & -2 \\ -3 & 0 \\ 3 & 5 \end{bmatrix}$, $B = \begin{bmatrix} 1 & 3 \\ 2 & -1 \end{bmatrix}$

Show solution ↓Hide solution ↑

Solution

Size check: $(3 \times 2)(2 \times 2) = 3 \times 2$ in both parts.

Part 1 (Exercise 5).

(a) Each $A\mathbf{b}_j$ is a linear combination of the columns of $A$ with weights from $\mathbf{b}_j$:

$$A\mathbf{b}_1 = 3\begin{bmatrix} -1 \\ 5 \\ 2 \end{bmatrix} - 2\begin{bmatrix} 2 \\ 4 \\ -3 \end{bmatrix} = \begin{bmatrix} -7 \\ 7 \\ 12 \end{bmatrix}, \qquad A\mathbf{b}_2 = -2\begin{bmatrix} -1 \\ 5 \\ 2 \end{bmatrix} + 1\begin{bmatrix} 2 \\ 4 \\ -3 \end{bmatrix} = \begin{bmatrix} 4 \\ -6 \\ -7 \end{bmatrix}$$$$AB = \begin{bmatrix} A\mathbf{b}_1 & A\mathbf{b}_2 \end{bmatrix} = \begin{bmatrix} -7 & 4 \\ 7 & -6 \\ 12 & -7 \end{bmatrix}$$

(b) Row–column rule:

$$AB = \begin{bmatrix} -1 \cdot 3 + 2(-2) & -1(-2) + 2 \cdot 1 \\ 5 \cdot 3 + 4(-2) & 5(-2) + 4 \cdot 1 \\ 2 \cdot 3 - 3(-2) & 2(-2) - 3 \cdot 1 \end{bmatrix} = \begin{bmatrix} -7 & 4 \\ 7 & -6 \\ 12 & -7 \end{bmatrix}$$

Part 2 (Exercise 6).

(a)

$$A\mathbf{b}_1 = 1\begin{bmatrix} 4 \\ -3 \\ 3 \end{bmatrix} + 2\begin{bmatrix} -2 \\ 0 \\ 5 \end{bmatrix} = \begin{bmatrix} 0 \\ -3 \\ 13 \end{bmatrix}, \qquad A\mathbf{b}_2 = 3\begin{bmatrix} 4 \\ -3 \\ 3 \end{bmatrix} - 1\begin{bmatrix} -2 \\ 0 \\ 5 \end{bmatrix} = \begin{bmatrix} 14 \\ -9 \\ 4 \end{bmatrix}$$$$AB = \begin{bmatrix} 0 & 14 \\ -3 & -9 \\ 13 & 4 \end{bmatrix}$$

(b)

$$AB = \begin{bmatrix} 4 \cdot 1 - 2 \cdot 2 & 4 \cdot 3 - 2(-1) \\ -3 \cdot 1 + 0 \cdot 2 & -3 \cdot 3 + 0(-1) \\ 3 \cdot 1 + 5 \cdot 2 & 3 \cdot 3 + 5(-1) \end{bmatrix} = \begin{bmatrix} 0 & 14 \\ -3 & -9 \\ 13 & 4 \end{bmatrix}$$

Both methods agree, as they must.

Exercise 3

(Adapted from Lay §2.1, Exercises 9 and 10)

1. Let $A = \begin{bmatrix} 2 & 5 \\ -3 & 1 \end{bmatrix}$ and $B = \begin{bmatrix} 4 & -5 \\ 3 & k \end{bmatrix}$. What value(s) of $k$, if any, will make $AB = BA$?
2. Let $A = \begin{bmatrix} 2 & -3 \\ -4 & 6 \end{bmatrix}$, $B = \begin{bmatrix} 8 & 4 \\ 5 & 5 \end{bmatrix}$, and $C = \begin{bmatrix} 5 & -2 \\ 3 & 1 \end{bmatrix}$. Verify that $AB = AC$ and yet $B \ne C$.

Show solution ↓Hide solution ↑

Solution

Part 1 (Exercise 9).

$$AB = \begin{bmatrix} 2 & 5 \\ -3 & 1 \end{bmatrix}\begin{bmatrix} 4 & -5 \\ 3 & k \end{bmatrix} = \begin{bmatrix} 8 + 15 & -10 + 5k \\ -12 + 3 & 15 + k \end{bmatrix} = \begin{bmatrix} 23 & -10 + 5k \\ -9 & 15 + k \end{bmatrix}$$$$BA = \begin{bmatrix} 4 & -5 \\ 3 & k \end{bmatrix}\begin{bmatrix} 2 & 5 \\ -3 & 1 \end{bmatrix} = \begin{bmatrix} 8 + 15 & 20 - 5 \\ 6 - 3k & 15 + k \end{bmatrix} = \begin{bmatrix} 23 & 15 \\ 6 - 3k & 15 + k \end{bmatrix}$$

The $(1,1)$ and $(2,2)$ entries already agree. Set the other entries equal:

• $(1,2)$: $-10 + 5k = 15 \Rightarrow k = 5$
• $(2,1)$: $-9 = 6 - 3k \Rightarrow 3k = 15 \Rightarrow k = 5$

Both conditions give the same value, so $AB = BA$ if and only if $k = 5$.

Part 2 (Exercise 10).

$$AB = \begin{bmatrix} 2 & -3 \\ -4 & 6 \end{bmatrix}\begin{bmatrix} 8 & 4 \\ 5 & 5 \end{bmatrix} = \begin{bmatrix} 16 - 15 & 8 - 15 \\ -32 + 30 & -16 + 30 \end{bmatrix} = \begin{bmatrix} 1 & -7 \\ -2 & 14 \end{bmatrix}$$$$AC = \begin{bmatrix} 2 & -3 \\ -4 & 6 \end{bmatrix}\begin{bmatrix} 5 & -2 \\ 3 & 1 \end{bmatrix} = \begin{bmatrix} 10 - 9 & -4 - 3 \\ -20 + 18 & 8 + 6 \end{bmatrix} = \begin{bmatrix} 1 & -7 \\ -2 & 14 \end{bmatrix}$$

So $AB = AC$, but clearly $B \ne C$. Cancellation fails for matrices.

Why it happens here: $\det A = 2(6) - (-3)(-4) = 12 - 12 = 0$, so $A$ is not invertible; you cannot “multiply both sides by $A^{-1}$” to cancel $A$. (If $A$ were invertible, $AB = AC$ would force $B = C$.)

Exercise 4

(Adapted from Lay §2.1, Exercise 11)

Let $A = \begin{bmatrix} 1 & 1 & 1 \\ 1 & 2 & 3 \\ 1 & 4 & 5 \end{bmatrix}$ and $D = \begin{bmatrix} 2 & 0 & 0 \\ 0 & 3 & 0 \\ 0 & 0 & 5 \end{bmatrix}$. Compute $AD$ and $DA$. Explain how the columns or rows of $A$ change when $A$ is multiplied by $D$ on the right or on the left. Find a $3 \times 3$ matrix $B$, not the identity matrix or the zero matrix, such that $AB = BA$.

Show solution ↓Hide solution ↑

Solution

$$AD = \begin{bmatrix} 1 & 1 & 1 \\ 1 & 2 & 3 \\ 1 & 4 & 5 \end{bmatrix}\begin{bmatrix} 2 & 0 & 0 \\ 0 & 3 & 0 \\ 0 & 0 & 5 \end{bmatrix} = \begin{bmatrix} 2 & 3 & 5 \\ 2 & 6 & 15 \\ 2 & 12 & 25 \end{bmatrix}$$$$DA = \begin{bmatrix} 2 & 0 & 0 \\ 0 & 3 & 0 \\ 0 & 0 & 5 \end{bmatrix}\begin{bmatrix} 1 & 1 & 1 \\ 1 & 2 & 3 \\ 1 & 4 & 5 \end{bmatrix} = \begin{bmatrix} 2 & 2 & 2 \\ 3 & 6 & 9 \\ 5 & 20 & 25 \end{bmatrix}$$

Interpretation:

• Right-multiplication by the diagonal $D$ ($AD$) multiplies each column of $A$ by the corresponding diagonal entry: column 1 by 2, column 2 by 3, column 3 by 5.
• Left-multiplication by $D$ ($DA$) multiplies each row of $A$ by the corresponding diagonal entry.

Since $AD \ne DA$, this is another example that order matters.

A matrix that commutes with $A$: any scalar multiple of the identity works, because $(rI)A = rA = A(rI)$. For example $B = 4I_3$:

$$AB = BA = 4A = \begin{bmatrix} 4 & 4 & 4 \\ 4 & 8 & 12 \\ 4 & 16 & 20 \end{bmatrix}$$

(Another valid choice is $B = A$ itself, or $B = A^2$, since $A \cdot A = A \cdot A$.)

Exercise 5

(Adapted from Lay §2.1, Exercise 17)

If $A = \begin{bmatrix} 1 & -2 \\ -2 & 5 \end{bmatrix}$ and $AB = \begin{bmatrix} -1 & 2 & -1 \\ 6 & -9 & 3 \end{bmatrix}$, determine the first and second columns of $B$.

Show solution ↓Hide solution ↑

Solution

Key idea: by the definition, $AB = \begin{bmatrix} A\mathbf{b}_1 & A\mathbf{b}_2 & A\mathbf{b}_3 \end{bmatrix}$. So column $j$ of $B$ solves $A\mathbf{x} = \text{(column } j \text{ of } AB)$.

First column: solve $A\mathbf{x} = \begin{bmatrix} -1 \\ 6 \end{bmatrix}$. Row reduce, using $R_2 \leftarrow R_2 + 2R_1$, then $R_1 \leftarrow R_1 + 2R_2$:

$$\left[\begin{array}{cc|c} 1 & -2 & -1 \\ -2 & 5 & 6 \end{array}\right] \sim \left[\begin{array}{cc|c} 1 & -2 & -1 \\ 0 & 1 & 4 \end{array}\right] \sim \left[\begin{array}{cc|c} 1 & 0 & 7 \\ 0 & 1 & 4 \end{array}\right] \quad\Rightarrow\quad \mathbf{b}_1 = \begin{bmatrix} 7 \\ 4 \end{bmatrix}$$

Second column: solve $A\mathbf{x} = \begin{bmatrix} 2 \\ -9 \end{bmatrix}$ with the same row operations:

$$\left[\begin{array}{cc|c} 1 & -2 & 2 \\ -2 & 5 & -9 \end{array}\right] \sim \left[\begin{array}{cc|c} 1 & -2 & 2 \\ 0 & 1 & -5 \end{array}\right] \sim \left[\begin{array}{cc|c} 1 & 0 & -8 \\ 0 & 1 & -5 \end{array}\right] \quad\Rightarrow\quad \mathbf{b}_2 = \begin{bmatrix} -8 \\ -5 \end{bmatrix}$$

Check: $A\mathbf{b}_1 = \begin{bmatrix} 7 - 8 \\ -14 + 20 \end{bmatrix} = \begin{bmatrix} -1 \\ 6 \end{bmatrix}$ ✓ and $A\mathbf{b}_2 = \begin{bmatrix} -8 + 10 \\ 16 - 25 \end{bmatrix} = \begin{bmatrix} 2 \\ -9 \end{bmatrix}$ ✓.

Since $\det A = 5 - 4 = 1 \ne 0$, you could also use $\mathbf{b}_j = A^{-1}(\text{column } j \text{ of } AB)$ with $A^{-1} = \begin{bmatrix} 5 & 2 \\ 2 & 1 \end{bmatrix}$. Both methods give the same answer.