G. Strang’s Linear Algebra IV-2

Reading notes

linear algebra
reading notes
Author

Alexandr Ten

Published

2026-Nov

The 2nd chapter from Gilbert Strang’s book on linear algebra (LA, Strang 2012) a deeper understanding of A\mathbf{x}=\mathbf{b} that is “in the heart of linear algebra” (p. 77). It connects to the preceeding Chapter 1 by stating LU factorization (via elimination) discussed at length there, provides only a partial perspective linear systems. Again, the authors intriges a reader trying to get a deep understanding of the subject. If it’s “in the heart”, it promises to be a parsimonious piece of knowledge helps you understand the whole. The chapter opens with introducing vector spaces and then builds on this to talk about subspaces and two special cases, the column space C(A), and the null space N(A) of matrix A.

Right off the bat, the discussion goes abstract, as Strang provided different examples of vector spaces. One of them is the space of all m \times n matrices. Another is the space of all functions f(x) defined over an interval. Surprisingly, vector spaces do not always to consist of vectors. To be a vector space, a set must contain things that can be (a) added together to produce the same kind of thing and (b) multiplied by a scalar to produce the same kind of thing.

Tip

More rigorously, the addition and multiplication operations, denoted \oplus and \otimes, respectively, must satisfy the following axioms in order for a triplet (V, \oplus, \otimes) to qualify as a vector space:

  1. \mathbf{v} \oplus \mathbf{w} = \mathbf{w} \oplus \mathbf{v} for \mathbf{v}, \mathbf{w} \in V
  2. (\mathbf{v} \oplus \mathbf{w}) \oplus \mathbf{u} = \mathbf{v} \oplus (\mathbf{w} \oplus \mathbf{u}) for \mathbf{u}, \mathbf{v}, \mathbf{w} \in V
  3. There exists a zero vector \mathbf{0} \in V such that \mathbf{v} \oplus \mathbf{0} = \mathbf{v} for every \mathbf{v} \in V
  4. For every \mathbf{v} \in V there exists an element -\mathbf{v} \in V such that \mathbf{v} \oplus (-\mathbf{v}) = \mathbf{0}
  5. 1 \otimes \mathbf{v} = \mathbf{v} for every \mathbf{v} \in V
  6. (ab) \otimes \mathbf{v} = a \otimes (b \otimes \mathbf{v}) for scalars a, b and \mathbf{v} \in V
  7. a \otimes (\mathbf{v} \oplus \mathbf{w}) = (a \otimes \mathbf{v}) \oplus (a \otimes \mathbf{w}) for a scalar a and \mathbf{v}, \mathbf{w} \in V
  8. (a + b) \otimes \mathbf{v} = (a \otimes \mathbf{v}) \oplus (b \otimes \mathbf{v}) for scalars a, b and \mathbf{v} \in V

It is fun to think of other examples and nonexamples of vector spaces. For instance, the set of people cannot be easily used to define a vector space. If we add two people together we get two people, not a new person. Even if we imagined that adding means sexual reproduction (ignoring the fact that 2 people of the same sex cannot reproduce), the 2nd axiom would not work: the child of Alice (\mathbf{v}) & Bob’s (\mathbf{w}) child and some person Charley (\mathbf{u}), call them Donna (~(\mathbf{v} \oplus \mathbf{w}) \oplus \mathbf{u}~), would not be the same as the child of Alice and Bob & Charley’s child (\mathbf{v} \oplus (\mathbf{w} \oplus \mathbf{u})). The set of real numbers \mathbb{R}, on the other hand, is a perfectly good vector space, as the familiar operations of simple addition and multiplication satisfy the 8 axioms.

The notion of a subspace is then introduced. A subspace inherits all the rules that made the containing space a vector space – Thus, a subspace is a vector space in its own right. Every subspace contains the special \boldsymbol{0} vector. This is because scalar multiplication must map to a vector in the same space. Multiplying by 0 lands on the zero vector by definition. All this leads up to the message that solutions to a system of equations A\mathbf{x} = \mathbf{b} are possible only for those vectors \mathbf{b} that live in the (sub)space spanned by columns of A. If there are fewer columns than rows, e.g., a 3 by 2 matrix, the columns of A are 3D vectors, so they live in the 3D space. However, they can only “walk” in a limited part of that space that passes through the origin. This is because the columns of A can be linearly combined to reach only a small part of the space they live in. Thus, there is only a “handful” of vectors \mathbf{b} for which \mathbf{x} satisfying A\mathbf{x} = \mathbf{b} exists and all of those vectors lie in the subspace spanned by the columns of A. Of course, this “handful” can be an infinite set, but this 2D (or 1D) sliver seems like nothing compared to the whole 3D space that extends infinitely in all 3 dimensions.

The metaphors “living in” and “walking” are helpful. We can think of the columns of an m by n matrix A as dwellers of \mathbb{R}^m. They do not want to get lost, so they keep their “home coordinates” relative to the shared origin. Individually, each can only move along the line through the origin and its home, forward or backward, as far as it likes. That way it is safer for them. However, they can talk to each other and discuss where they could go. Column \mathbf{u} could say something like “I can go 3 scalar steps forward”, to which column \mathbf{w} could add “and I could then go 5 scalar steps backward from where you (\mathbf{u}) stop, and we would reach 3\mathbf{u} + (-5)\mathbf{w}.” With these metaphors, a “solution” to an equation is a particular collection of directions that the columns can use (\mathbf{x}) to reach a destination \mathbf{b}. Sometimes, some destinations are not reachable. Sometimes, the columns live on the same line, so they can go to an even smaller number of places.

References

Strang, Gilbert. 2012. Linear Algebra and Its Applications. 4th ed. Cengage Learning.