Skip to main content

Euclidean vector spaces

Section 4.1 Subspaces

We return now to our main object of study: vector spaces. Our foray into the theory of matrices will prove to be useful in this regard in two ways: on the one hand, matrix spaces \(M_{mn}\) are themselves interesting examples of vector spaces; on the other hand, matrices serve as an essential computational tool for describing and investigating general vector spaces.
In this section we will study subspaces, which are special subsets of vector spaces that respect the defining structure of a vector spaces: namely, the two vector operations. DefinitionΒ 4.1.1 makes precise what we mean here by β€˜respect’.
Subspaces arise naturally in any setting where vector spaces are at play, and are closely connected to solutions to linear systems. As we will see in TheoremΒ 4.1.7, subspaces of vector spaces are themselves examples of vector spaces, furnishing us with yet more interesting examples of vector spaces.

Subsection 4.1.1 Definition of subspace

Definition 4.1.1. Subspace.

Let \(V\) be a vector space. A subset \(W\subseteq V\) is a subspace of \(V\) if the following conditions hold:
  1. \(W\) contains the zero vector.
    We have \(\boldzero\in W\text{.}\)
  2. \(W\) is closed under addition.
    For all \(\boldv_1,\boldv_2\in V\text{,}\) if \(\boldv_1,\boldv_2\in W\text{,}\) then \(\boldv_1+\boldv_2\in W\text{.}\) Using logical notation:
    \begin{equation*} \boldv_1,\boldv_2\in W\implies \boldv_1+\boldv_2\in W\text{.} \end{equation*}
  3. \(W\) is closed under scalar multiplication.
    For all \(c\in \R\) and \(\boldv\in V\text{,}\) if \(\boldv\in W\text{,}\) then \(c\boldv\in W\text{.}\) In logical notation:
    \begin{equation*} \boldv\in W\Rightarrow c\boldv\in W\text{.} \end{equation*}

Example 4.1.2.

Let \(V=\R^2\) and let
\begin{equation*} W=\{(t,t)\in\R^2 \colon t\in\R\}\text{.} \end{equation*}
Prove that \(W\) is a subspace.
Solution.
We must show properties (i)-(iii) hold for \(W\text{.}\)
  1. The zero element of \(V\) is \(\boldzero=(0,0)\text{,}\) which is certainly of the form \((t,t)\text{.}\) Thus \(\boldzero\in W\text{.}\)
  2. We must prove the implication \(\boldv_1, \boldv_2\in W\Rightarrow \boldv_1+\boldv_2\in W\text{.}\)
    \begin{align*} \boldv_1,\boldv_2\in W\amp \Rightarrow\amp \boldv_1=(t,t), \boldv_2=(s,s) \text{ for some \(t,s\in\R\) }\\ \amp \Rightarrow\amp \boldv_1+\boldv_2=(t+s,t+s)\\ \amp \Rightarrow\amp \boldv_1+\boldv_2\in W\text{.} \end{align*}
  3. We must prove the implication \(\boldv\in W\Rightarrow c\boldv\in W\text{,}\) for any \(c\in \R\text{.}\) We have
    \begin{align*} \boldv\in W\amp \Rightarrow\amp \boldv=(t,t)\\ \amp \Rightarrow\amp c\boldv=(ct,ct)\\ \amp \Rightarrow\amp c\boldv\in W \end{align*}

Example 4.1.3.

Let \(V=\R^n\) and let
\begin{equation*} W=\{(x,y)\in \R^2\colon x, y\geq 0\}\text{.} \end{equation*}
Is \(W\) a vector space? Decide which of the of properties (i)-(iii) in DefinitionΒ 4.1.1 (if any) are satisfied by \(W\text{.}\)
Solution.
  1. Clearly \(\boldzero=(0,0)\in W\text{.}\)
  2. Suppose \(\boldv_1=(x_1,y_1), \boldv_2=(x_2,y_2)\in W\text{.}\) Then \(x_1, x_2, y_1, y_2\geq 0\text{,}\) in which case \(x_1+x_2, y_1+y_2\geq 0\text{,}\) and hence \(\boldv_1+\boldv_2\in W\text{.}\) Thus \(W\) is closed under addition.
  3. The set \(W\) is not closed under scalar multiplication. Indeed, let \(\boldv=(1,1)\in W\text{.}\) Then \((-2)\boldv=(-2,-2)\notin W\text{.}\)

Example 4.1.5. Video example: deciding if \(W\subseteq V\) is a subspace.

Figure 4.1.6. Video: deciding if \(W\subseteq V\) is a subspace
If \(W\) is a subspace of a vector space \(V\text{,}\) then it inherits a vector space structure from \(V\) by simply restricting the vector operations defined on \(V\) to the subset \(W\text{.}\)

Proof.

Since \(\boldw_1+\boldw_2\in W\) for all \(w_1,w_2\in W\text{,}\) the vector addition on \(V\) gives rise by restriction to a well-defined operation on \(W\text{;}\) similarly, since \(c\boldw\in W\) for all \(c\in \R\) and \(\boldw\in W\text{,}\) the scalar multiplication operation on \(V\) gives rise by restriction to a well-defined scalar multiplication on \(W\text{.}\)
By AxiomΒ i, the zero vector \(\boldzero\) of \(V\) is an element of \(W\text{.}\) Since this element satisfies \(\boldzero+\boldv=\boldv\) for all \(\boldv\in V\text{,}\) and since \(W\subseteq V\text{,}\) it also satisfies \(\boldzero+\boldw=\boldw\) for all \(\boldw\in W\text{.}\) Thus \(\boldzero\) acts as a zero vector for the subspace \(W\text{.}\)
It is important to understand how AxiomsΒ i–iii of DefinitionΒ 4.1.1 come into play here. Without them we would not be able to say that restricting the vector operations of \(V\) to elements of \(W\) actually gives rise to well-defined operations on \(W\text{.}\) To be well-defined the operations must output elements that lie not just in \(V\text{,}\) but in \(W\) itself. This is precisely what being closed under addition and scalar multiplication guarantees.
Once we know restriction gives rise to well-defined operations on \(W\text{,}\) verifying the axioms of DefinitionΒ 1.1.4 mostly amounts to observing that if a condition is true for all \(\boldv\) in \(V\text{,}\) it is certainly true for all \(\boldv\) in the subset \(W\text{.}\)
The β€œexistential axioms” (iii) and (iv) of DefinitionΒ 1.1.4, however, require special consideration. By definition, a subspace \(W\) contains the zero vector of \(V\text{,}\) and clearly this still acts as the zero vector when we restrict the vector operations to \(W\text{.}\) What about vector inverses? We know that for any \(\boldv\in W\) there is a vector inverse \(-\boldv\) lying somewhere in \(V\text{.}\) We must show that in fact \(-\boldv\) lies in \(W\text{:}\) i.e. we need to show that the operation of taking the vector inverse is well-defined on \(W\text{.}\) We prove this as follows:
\begin{align*} \boldv\in W \amp\implies (-1)\boldv\in W \amp (\knowl{./knowl/xref/d_subspace.html}{\text{Definition 4.1.1}}, \text{(iii) } )\\ \amp\implies -\boldv\in W \amp (\knowl{./knowl/xref/th_vectorspace_props.html}{\text{Theorem 1.1.9}}, (iii)) \text{.} \end{align*}
We now know how to determine whether a given subset of a vector space is in fact a subspace. We are also interested in means of constructing subspaces from some given ingredients. The result below tells us that taking the intersection of a given collection of subspaces results in a subspace.

Proof.

Remark 4.1.9. Unions of subspaces.

While the intersection of subspaces is again a subspace, the same is not true for unions of subspaces.
For example, take \(V=\R^2\text{,}\) \(W_1=\{(t,t)\colon t\in\R\}\) and \(W_2=\{(t,-t)\colon t\in\R\}\text{.}\) Then each \(W_i\) is a subspace, but their union \(W_1\cup W_2\) is not.
Indeed, observe that \(\boldw_1=(1,1)\in W_1\subset W_1\cup W_2\) and \(\boldw_2=(1,-1)\in W_2\subset W_1\cup W_2\text{,}\) but \(\boldw_1+\boldw_2=(2,0)\notin W_1\cup W_2\text{.}\) Thus \(W_1\cup W_2\) is not closed under addition. (Interestingly, it is closed under scalar multiplication.)

Subsection 4.1.2 Subspaces of \(\R^n\)

TheoremΒ 4.1.11 gives a convenient method of producing a subspace \(W\) of \(\R^n\text{:}\) namely, given any \(m\times n\) matrix \(A\text{,}\) the set
\begin{equation*} W=\{\boldx\in \R^n\colon A\boldx=\boldzero\} \end{equation*}
of all solutions to the homogeneous linear system \(A\boldx=\boldzero\) is guaranteed to be a subspace of \(\R^n\text{.}\) We call this set the null space of the matrix \(A\text{.}\)

Definition 4.1.10. Null space of matrix.

Let \(A\in M_{mn}\text{.}\) The null space of \(A\text{,}\) denoted \(\NS A\text{,}\) is the set of all solutions to the matrix equation
\begin{equation} A\boldx=\underset{m\times 1}{\boldzero}\text{.}\tag{4.1} \end{equation}
In other words,
\begin{equation*} \NS A=\{\boldx\in \R^n\colon A\boldx=\boldzero\}\text{.} \end{equation*}
Equivalently, thinking in terms of linear systems, \(\NS A\) is the set of solutions to the homogeneous linear system represented by (4.1).

Proof.

Following the two-step technique, we first show that the zero vector \(\boldzero=(0,0,\dots, 0)\) of \(\R^n\) lies in \(\NS A\text{.}\) This is clear, since
\begin{equation*} A\underset{n\times 1}{\boldzero}=\underset{m\times 1}{\boldzero}\text{.} \end{equation*}
Next, we show that for any \(\boldx_1, \boldx_2\in \R^n\) and any \(c_1, c_2\in \R\) we have
\begin{equation*} \boldx_1, \boldx_2\in \NS A\implies c_1\boldw_1+c_2\boldw_2\in \NS A\text{.} \end{equation*}
If \(\boldx_1, \boldx_2\in \NS A\text{,}\) then we have \(A\boldx_1=A\boldx_2=\boldzero\text{,}\) by definition. It then follows that the vector \(c_1\boldx_1+c_2\boldx_2\) satisfies
\begin{align*} A(c_1\boldx_1+c_2\boldx_2) \amp= c_1A\boldx_1+c_2A\boldx_2 \amp (\knowl{./knowl/xref/th_matrix_alg_props.html}{\text{3.2.1}}) \\ \amp c_1A\boldzero_m+c_2\boldzero_m \amp (\boldx_1, \boldx_2\in W) \\ \amp = \boldzero\text{.} \end{align*}
Since \(A(c\boldx_1+d\boldx_2)=\boldzero\text{,}\) we have \(c_1\boldx_1+c_2\boldx_2\in \NS A\text{,}\) as desired.

Remark 4.1.12. Alternative subspace method.

TheoremΒ 4.1.11 provides an alternative way of showing that a subset \(W\subseteq \R^n\text{:}\) namely, find an \(m\times n\) matrix \(A\) for which we have \(W=\{\boldx\in \R^n\colon A\boldx=\boldzero\}\text{.}\) This is often much faster than using the two-step technique.

Example 4.1.13. Subspace as null space.

Define the subset \(W\) of \(\R^3\) as
\begin{equation*} W=\{(x,y,z)\in \R^3\colon x+2y+3z=x-y-z=0\}\text{.} \end{equation*}
  1. Prove that \(W\) is a subspace by identifying it as the set of solutions to a homogeneous matrix equation.
  2. Use (a) and Gaussian elimination to compute a parametric description of \(W\text{.}\)
Solution.
  1. It is easy to see that
    \begin{equation*} W=\{\boldx\in \R^n\colon A\boldx=\boldzero\} \end{equation*}
    where
    \begin{equation*} A=\begin{amatrix}[rrr]1\amp 2\amp 3\\ 1\amp -1\amp -1 \end{amatrix}\text{.} \end{equation*}
    We conclude \(W\) is a subspace.
  2. The augmented matrix \(\begin{amatrix}[c|c]A\amp\boldzero \end{amatrix}\) row reduces to
    \begin{equation*} U=\begin{amatrix}[rrr|r]\boxed{1}\amp 2\amp 3\amp 0\\ 0 \amp \boxed{1}\amp 1\amp 0 \end{amatrix}\text{.} \end{equation*}
    Following ProcedureΒ 2.3.6 we conclude that
    \begin{equation*} W=\{(-t,-2t,t)\colon t\in \R\}\text{.} \end{equation*}
    Geometrically this is the line in \(\R^3\) passing through \((0,0,0)\) with direction vector \((1,2,-1)\text{.}\)

Warning 4.1.14. Subspace as null space.

As convenient as the method described in RemarkΒ 4.1.12 and illustrated in ExampleΒ 4.1.13 may be, bear in mind that it cannot always be used. Indeed, by definition the null space of an \(m\times n\) matrix is a subset of \(\R^n\text{.}\) Thus this method can only be employed when the ambient vector space is \(\R^n\text{.}\) Don’t forget that there are other vector spaces besides \(\R^n\text{.}\) Indeed, in SubsectionΒ 4.1.3 we consider subspaces of matrix vector spaces \(M_{mn}\text{.}\) In this setting, our null space trick does not apply.
Let \(A\) be an \(m\times n\) matrix. If \(\boldb\in \R^m\) is nonzero, then the set of solutions \(S\) to \(A\boldx=\boldb\) is not a subspace of \(\R^n\text{,}\) and for a very simple reason: since
\begin{equation*} A\underset{n\times 1}{\boldzero}=\underset{m\times 1}{\boldzero}\ne \boldb\text{,} \end{equation*}
we see that \(\underset{n\times 1}{\boldzero}\notin S\text{,}\) and thus \(S\) is not a subspace. Thus, thinking in terms of linear systems, we see that while the set of solutions to a homogenous linear system constitutes a subspace, the set of solutions to a nonhomogeneous system does not. On the other hand, as articulated by TheoremΒ 4.1.15, the set of solutions to a nonhomogeneous linear system can be thought of as a translate of a vector space.

Proof.

  1. If \(\boldb=\boldzero\text{,}\) then \(S=\NS A\text{,}\) and this is a subspace by TheoremΒ 4.1.11. If \(\boldb\ne \boldzero\text{,}\) then \(\boldzero\notin S\text{,}\) and hence \(S\) is not a subspace.
  2. Let \(\boldx_p\) satisfy (4.2). We show that
    \begin{equation*} S=\boldx_p+\NS A \end{equation*}
    by showing the two inclusions
    \begin{align*} \boldx_p+\NS A\amp \subseteq S \amp S\amp \boldx_p+\NS A\text{.} \end{align*}
    If \(\boldv\in \boldx_p+\NS A\text{,}\) then we have \(\boldv=\boldx_p+\boldw\) for some \(\boldw\in \NS A\text{,}\) in which case
    \begin{align*} A\boldv \amp = A(\boldx_p+\boldw)\\ \amp = A\boldx_p+A\boldw\\ \amp = \boldb+\boldzerp \amp (\boldx_p\in S, \boldw\in \NS A)\\ \amp = \boldb\text{.} \end{align*}
    This shows that if \(\boldv\in \boldx_p+\NS A\text{,}\) then \(\boldv\in S\text{,}\) and thus that \(\boldx_p+A\subseteq S\text{.}\) For the other inclusion, if \(\boldv\in S\text{,}\) then we have
    \begin{align*} A(\boldx_p-\boldv) \amp = A\boldx_p-A\boldv\\ \amp = \boldb-\boldb\\ \amp = \boldzero\text{,} \end{align*}
    showing that \((\boldx_p-\boldv)\in \NS A\text{.}\) But then we have
    \begin{equation*} \boldv=\boldx_p+\underset{\boldw}{(\boldx_p-\boldv)}\text{,} \end{equation*}
    where \(\boldw=(\boldx_p-\boldv)\in \NS A\text{.}\) Thus \(\boldv\in \boldx_p+\NS A\text{,}\) showing that \(S\subseteq \boldx_p+\NS A\text{.}\)

Sage example 10. Solving matrix equations.

Let’s use Sage and CorollaryΒ 4.1.16 to find the set of solutions \(S\subseteq \R^5\) to the matrix equation
\begin{equation} \begin{amatrix}[rrrrr] 0\amp 0\amp -2\amp 0\amp 7\\ 2\amp 4\amp -10\amp 6\amp 12\\ 2\amp 4\amp -5\amp 6\amp -5 \end{amatrix} \begin{amatrix}[c] x_1\\ x_2\\ x_3\\ x_4\\ x_5 \end{amatrix}= \begin{amatrix}[r] 12\\ 28\\ -1 \end{amatrix}\text{.}\tag{4.4} \end{equation}
This is the matrix equation form of the linear system we investigated in Sage exampleΒ 3. The method solve_right can be used to find a particular solution \(\boldx_p\) to (4.4).
We get the entire set of solutions \(S\) by translating \(\NS A\) by the particular solution \(\boldx_p\text{:}\)
\begin{equation*} S=\{\boldx_p+\boldu\colon \boldu\in \NS A\}=\boldx_p+\NS A\text{.} \end{equation*}
We can illustrate this in Sage by taking random elements of \(\NS A\) (computed using right_kernel), adding them to xp, and verifying that the result is a solution to (4.4). Each time you evaluate the cell below, a randomly generated element of \(S\) will be outputted.
You may wonder just how random these elements of \(S\) are, considering that the entries always seem to be integers! Indeed, soliciting information about NS from Sage, we see that it has the structure of a β€œfree module” defined over the the β€œInteger Ring”.
Without getting too far into the weeds, this is a result of our initial definition of \(A\) using Matrix(). Without further information, Sage interprets this as a matrix with integer coefficients, as opposed to real coefficients. All further computations (e.g., xp and NS) are done in a similar spirit. More precisely, the object NS generated by Sage consists of all integer linear combinations of the two rows in the β€œechelon basis matrix” displayed in the cell above. The next cell shows you how things change when we alert Sage to the fact that we are dealing with matrices over the reals. The only change is adding RR to Matrix(), which specifies that matrix coefficients should be understood as real numbers.

Remark 4.1.17. Hyperplanes and subspaces.

Recall that a hyperplane is the set of solutions \(H\subseteq \R^n\) to a linear system of the form
\begin{equation} a_1x_1+a_2x_2+\cdots +a_nx_n=b\text{,}\tag{4.5} \end{equation}
where \(a_i\ne 0\) for some \(i\text{.}\) In terms of TheoremΒ 4.1.15, \(H\) is just the set of solutions to the matrix equation \(A\boldx=\boldb\text{,}\) where
\begin{align*} A \amp =\begin{bmatrix} a_1\amp a_2\amp \dots \amp a_n \end{bmatrix} \amp \boldb=\begin{bmatrix} b \end{bmatrix} \text{.} \end{align*}
It follows from TheoremΒ 4.1.15 that \(H\) is a subspace if and only if \(b=0\text{:}\) i.e., if and only if \(H\) passes through the origin. Furthermore, if \(b\ne 0\text{,}\) we have
\begin{equation*} H=\boldx_p+H_0\text{,} \end{equation*}
where \(\boldx_p\) is any solution to (4.5), and \(H_0\) is the set of solutions to the corresponding homogeneous equation
\begin{equation} a_1x_1+a_2x_2+\cdots +a_nx_n=0\text{.}\tag{4.6} \end{equation}
In other words, although it is not true in general that every hyperplane \(H\subseteq \R^n\) is a subspace (since it may not pass through the origin), it is true that \(H=\boldx_p+H_0\) is a translate of a hyperplane \(H_0\) that is a subspace (since \(H_0\) passes through the origin).

Subsection 4.1.3 Important subspaces of \(M_{nn}\)

In The invertibility theorem we met three families of square matrices: namely, the diagonal, upper triangular, and lower triangular matrices. (See DefinitionΒ 3.4.7). We now introduce three more naturally occurring families. Before doing so, we give an official definition of the trace function.

Definition 4.1.18. Trace of a matrix.

Let \(A=[a_{ij}]\) be an \(n\times n\) matrix. The trace of \(A\text{,}\) denoted \(\tr A\) is defined as the sum of the diagonal entries of \(A\text{:}\) i.e.,
\begin{equation*} \tr A=a_{11}+a_{22}+\cdots +a_{nn}\text{.} \end{equation*}

Definition 4.1.19. Trace-zero, symmetric, and skew-symmetric.

Fix an integer \(n\geq 1\text{.}\)
  1. A matrix \(A\in M_{nn}\) is said to be a trace-zero matrix if \(\tr A=0\text{.}\)
  2. A matrix \(A\in M_{nn}\) is symmetric if \(A^T=A\text{:}\) equivalently, if \([A]_{ij}=[A]_{ji}\) for all \(1\leq i,j\leq n\text{.}\)
  3. A matrix \(A\in M_{nn}\) is skew-symmetric if \(A^T=-A\text{:}\) equivalently, if \([A]_{ij}=-[A]_{ji}\) for all \(1\leq i,j\leq n\text{.}\)

Example 4.1.20. Trace-zero symmetric, and skew-symmetric \(2\times 2\) matrices.

The set \(W_1\) of all trace-zero \(2\times 2\) matrices can be described as
\begin{equation*} W_1=\left\{ \begin{amatrix}[rr]a\amp b\\ c\amp -a \end{amatrix}\colon a,b,c\in \R\right\}\text{.} \end{equation*}
The set \(W_2\) of all symmetric \(2\times 2\) matrices can be described as
\begin{equation*} W_2=\left\{ \begin{amatrix}[rr]a\amp b\\ b\amp c \end{amatrix}\colon a,b,c\in \R\right\}\text{.} \end{equation*}
The set \(W_3\) of all skew-symmetric \(2\times 2\) matrices can be described as
\begin{equation*} W_3=\left\{ \begin{amatrix}[rr]0\amp a\\ -a\amp 0 \end{amatrix}\colon a,b\in \R\right\}\text{.} \end{equation*}

Remark 4.1.21.

Assume \(A\) is a skew-symmetric \(n\times n\) matrix. By definition, for all \(1\leq i\leq n\) we must have \([A]_{ii}=-[A]_{ii}\text{.}\) It follows that \([A]_{ii}=0\) for all \(1\leq i\leq n\text{.}\) Thus the diagonal entries of a skew-symmetric matrix are always equal to 0.
It will come as no surprise that all of the afore mentioned matrix families are in fact subspaces of \(M_{nn}\text{.}\)

Proof.

Exercises 4.1.4 Exercises

Exercise Group.

For each subset \(W\) of \(\R^2\) described below: (a) sketch \(W\) as a region of \(\R^2\text{,}\) and (b) determine whether \(W\) is a subspace. Justify your answer either with a proof or explicit counterexample.

Exercise Group.

Determine whether the subset \(W\) of \(M_{nn}\) described is a subspace of \(M_{nn}\text{.}\) Justify your answer either with a proof or explicit counterexample.
6.
Fix a matrix \(B\in M_{nn}\) and define \(W=\{A\in M_{nn}\colon AB=BA\}\text{,}\) the set of matrices that commute with \(B\text{.}\)

Exercise Group.

For each given subset \(W\subseteq \R^n\text{:}\) (a) show that \(W\) is a subspace by identifying it with the set of solutions to a matrix equation, and (b) give a parametric description of \(W\text{.}\)
8.
\(W=\{(x_1,x_2,x_3,x_4)\colon x_1-x_4=x_2+x_3=x_1+x_2+x_3=0\}\subseteq \R^4\)

Nonhomogeneous solutions related to homogeneous solutions.

For each \(m\times n\) matrix \(A\) and vector \(\boldb\in \R^m\text{:}\)
  1. Find a particular solution \(\boldx_p\) to \(A\boldx=\boldb\text{.}\)
  2. Find all solutions to the corresponding homogeneous matrix equation \(A\boldx=\boldzero\text{.}\)
  3. Use (a), (b), and CorollaryΒ 4.1.16 to describe all solutions to \(A\boldx=\boldb\text{.}\)
12.
\begin{equation*} A=\begin{amatrix}[rrrr]1\amp 2\amp 1\amp 1\\ 1\amp 1\amp 2\amp 3 \end{amatrix}, \boldb=(3,2) \end{equation*}
13.
\begin{equation*} A=\begin{amatrix}[rrr]1\amp 1\amp -3 \\ 3\amp -1\amp -1 \\ 1\amp 0\amp -1 \end{amatrix}, \boldb=(2,1,-4) \end{equation*}
14.
\begin{equation*} A=\begin{amatrix}[rrr]1\amp 2\amp 1 \\ 1\amp 1\amp -3 \\ 1\amp 0\amp -1 \end{amatrix}, \boldb=(1,1,1) \end{equation*}