A brief survey of linear algebra

Tom Carter

http://cogs.csustan.edu/˜ tom/linear-algebra

Santa Fe Institute Complex Systems Summer School

June, 2001

Our general topics: ←

⊚ Why linear algebra ⊚ Vector spaces (ex) ⊚ Examples of vector spaces (ex) ⊚ Subspaces (ex) ⊚ Linear dependence and independence (ex) ⊚ Span of a set of vectors (ex) ⊚ Basis for a vector space (ex) ⊚ Linear transformations (ex) ⊚ Morphisms – mono, epi, and iso (ex) ⊚ Linear operators (ex) ⊚ Normed linear spaces (ex) ⊚ Eigenvectors and eigenvalues (ex) ⊚ Change of basis (ex) ⊚ Trace and determinant (ex) ⊚ References

(ex): exercises.

Why linear algebra ←

• Linear models of phenomena are pervasive throughout science. The techniques of linear algebra provide tools which are applicable in a wide variety of contexts.

Beyond that, linear algebra courses are often the transition from lower division mathematics courses such as calculus, probability/statistics, and elementary differential equations, which typically focus on specific problem solving techniques, to the more theoretical axiomatic and proof oriented upper division mathematics courses.

I am going to stay with a generally abstract, axiomatic presentation of the basics of linear algebra. (But I’ll also try to provide some practical advice along the way …:-)

Vector spaces ←

• The first thing we need is a field 𝔽 of coefficients for our vector space. The most frequently used fields are the real numbers ℝ and the complex numbers ℂ. A field 𝔽 = (𝔽, +, ∗, −, ^{−1}, 0, 1) is a mathematical object consisting of a set of elements (𝔽), together with two binary operations (+, ∗), two unary operations (−, ^{−1}), and two distinguished elements 0 and 1 of 𝔽, which satisfy the fundamental properties:

1. 𝔽 is closed under the four operations:

+ : 𝔽 × 𝔽 → 𝔽
∗ : 𝔽 × 𝔽 → 𝔽
− : 𝔽 → 𝔽
−1 : 𝔽′ → 𝔽′ 𝔽′ = {a ∈ 𝔽|a ≠ 0}

Of course, we usually write

a + b, a ∗ b, −a, and a^{−1} instead of +(a, b), ∗(a, b), −(a), and ^{−1}(a).

2. ‘+’ and ‘∗’ are commutative and associative, and satisfy the distributive property. That is, for a, b, c ∈ 𝔽:

a + b = b + a
a ∗ b = b ∗ a
(a + b) + c = a + (b + c)
(a ∗ b) ∗ c = a ∗ (b ∗ c)
a ∗ (b + c) = a ∗ b + a ∗ c

3. 0 is the identity element and ‘−’ is the inverse for addition. 1 is the identity element and ‘a^{−1}’ is the inverse for multiplication. That is, for a ∈ 𝔽:

a + 0 = a
a + (−a) = 0
a ∗ 1 = a
a ∗ (a^{−1}) = 1  (a ≠ 0)

4. Although we won’t need it for most of linear algebra, I’ll mention that ℝ and ℂ are both complete (Cauchy sequences have limits), and ℝ is fully ordered (a < b or b < a or a = b for all a, b ∈ ℝ).

5. As needed, we will identify ℝ as a subfield of ℂ, and we will typically write elements of ℂ as a + bi, where a and b are real and i^{2} = −1.

6. Although ℝ and ℂ are the most frequently used coefficient fields, there are many other fields such as ℚ, the rationals, and the finite fields ℤ/pℤ for p a prime.

• If we leave out the requirement that a ∗ b = b ∗ a, we get what are called skew fields. An important example of a skew field is ℍ, the quaternions (also called the hamiltonians) which contains additional square roots of −1, i^{2} = j^{2} = k^{2} = −1, and ij = k, jk = i, ki = j, but ij = −ji, jk = −kj, and ki = −ik. We typically write quaternions either in the form a + bi + cj + dk with a, b, c, d ∈ ℝ, or α + βj with α, β ∈ ℂ.

• We are then ready for the definition. A vector space V = (V, 𝔽, +, −, ∗, 0) over a field 𝔽 is a set V of elements (called vectors) together with a distinguished element 0, two binary operations, and a unary operation:

+ : V × V → V
− : V → V
∗ : 𝔽 × V → V

For u, v, w ∈ V, and a, b ∈ 𝔽, these operations satisfy the properties:

v + w = w + v
(u + v) + w = u + (v + w)
v + 0 = v
v + (−v) = 0
1 ∗ v = v
a ∗ (u + v) = (a ∗ u) + (a ∗ v)
(a + b) ∗ v = (a ∗ v) + (b ∗ v)
(a ∗ b) ∗ v = a ∗ (b ∗ v)

The elements of the field 𝔽 are called the scalars, or coefficients of the vector space.

• From the basic properties listed above, we can prove a variety of additional properties, such as:

0 ∗ v = 0
a ∗ 0 = 0
(−1) ∗ v = −v

We can also prove that the additive identity 0 is unique, as is the additive inverse −v.

• We will usually simplify the notation, and write av instead of a ∗ v. Furthermore, although it is important to distinguish the scalar 0 from the vector 0, we will typically write the vector in the simple form 0.

• If 𝔽 = ℝ, we call V a real vector space, and typically write the scalars as a, b, c. If 𝔽 = ℂ, we call V a complex vector space, and often write the scalars as α, β, γ.

Exercises: Vector spaces←

1. Using the basic properties listed above, prove the additional properties:

(a) 0 ∗ v = 0

(b) a ∗ 0 = 0

(c) (−1) ∗ v = −v

(d) −(−v) = v

(e) For a ∈ 𝔽 and v ∈ V, av = 0 iff a = 0 or v = 0.

2. Prove that the additive identity 0 is unique.

3. Prove that the additive inverse −v is unique.

Examples of vector spaces ←

• The first main example of a real vector space is ℝ^{n}, the Cartesian product of n copies of the real line. An element of ℝ^{n} looks like (a_{1}, a_{2}, …, a_{n}). When we add two vectors, we get

(a_{1}, a_{2}, …, a_{n}) + (b_{1}, b_{2}, …, b_{n})

= (a_{1} + b_{1}, a_{2} + b_{2}, …, a_{n} + b_{n}).

When we multiply by a scalar a ∈ ℝ, we get

a ∗ (a_{1}, a_{2}, …, a_{n}) = (aa_{1}, aa_{2}, …, aa_{n}).

• Similary, we have the complex vector space ℂ^{n}, with vector addition and scalar multiplication defined the same way.

• Note that we can also think of ℂ^{n} as a real vector space, if we restrict the scalars to ℝ.

• Let 𝔽^{∞} = {(a_{0}, a_{1}, …)}, with

(a_{0}, a_{1}, …) + (b_{0}, b_{1}, …)

= (a_{0} + b_{0}, a_{1} + b_{1}, …), and a ∗ (a_{0}, a_{1}, …) = (a ∗ a_{0}, a ∗ a_{1}, …). This is a vector space over 𝔽.

• For a scalar field 𝔽, we can define 𝔽[x] to be the set of all polynomials with formal variable x, and coefficients in 𝔽. This is a vector space over 𝔽, the coefficient field. Each polynomial is a vector. Vector addition is just polynomial adddition, and scalar multiplication just multiplies each coefficient in the polynomial by the scalar:

a ∗ (a_{0} + a_{1}x^{1} + … a_{n}x^{n})
= aa_{0} + aa_{1}x^{1} + … aa_{n}x^{n}.

• For a scalar field 𝔽, let 𝔽^{n}[x] be the set of all polynomials of degree ≤ n with coefficients in 𝔽. We can also let 𝔽^{∞}[x] be the vector space of (formal) power series over 𝔽. These are also vector spaces over 𝔽.

• Let C^{0}(ℝ) be the set of all continuous functions with domain and range the real numbers. That is: C^{0}(ℝ) = {f : ℝ → ℝ | f is continuous }. We can define an addition operation on this set. To specify the sum of two functions, we must specify the sum at each point in the domain. If f, g ∈ C^{0}(ℝ), we define f + g for each x by

(f + g)(x) = f(x) + g(x).

We define scalar multiplication pointwise also: (a ∗ f)(x) = a ∗ (f(x)).

C^{0}(ℝ) thus becomes a real vector space, where each continuous function is a vector in the space.

• Let C^{n}(ℝ) be the set of all continuous real functions whose first n derivatives are also continuous, and define addition and scalar multiplication pointwise. Similarly, we can define C^{∞}(ℝ) as functions with all derivatives continuous. Again we get real vector spaces.

• Let C^{0}[0, 1] be the set of all continuous functions with domain the closed interval [0, 1], and range ℝ. We can also define C^{n}[0, 1] to be those functions with first n derivatives continuous, and C^{∞}[0, 1] with all derivatives continuous. We can also generalize to subsets of ℝ other than the interval [0, 1]. With pointwise addition and scalar multiplication, these are each real vector spaces.

• We get can get similar complex vector spaces if we use ℂ instead of ℝ in examples like those above.

Exercises: Examples of vector spaces ←

1. Show that each of the examples listed in this section is a vector space.

2. Consider the set of points in ℝ^{2} of the forms (x, 0), x ∈ ℝ, and (0, y), y ∈ ℝ (i.e., the union of the X-axis and the Y-axis). Show that with the usual vector addition in ℝ^{2}, this is not a vector space over ℝ.

3. Consider the set ℝ^{2} with usual vector addition:

(x_{1}, y_{1}) + (x_{2}, y_{2}) = (x_{1} + x_{2}, y_{1} + y_{2}),

but with “scalar multiplication” given by

a ∗ (x, y) = (ax, y)

for a ∈ ℝ. Show that this is not a vector space over ℝ.

Subspaces ←

• If V is a vector space over 𝔽, a subset U ⊂ V is called a vector subspace of V if U is a vector space over 𝔽 using the same vector addition and scalar multiplication as in V. Often, in context, we will just call U a subspace of V.

• Most often, given a subset U ⊂ V, we will just let U inherit the vector addition and scalar multiplication operations from V. Hence, most of the vector space properties will come for free. What may not come for free, however, is whether the subset U is closed under the operations.

Thus, when we inherit the operations from V, we will have that

+ : U × U → V,

and

∗ : 𝔽 × U → V,

whereas, what we need is

+ : U × U → U,

and

∗ : 𝔽 × U → U.

In order for U to be a vector subspace, we must be sure that doing the operations will leave us in the subset U.

• Examples of subspaces:

1. {0} is a subspace of V for any vector space V.

2. If m ≤ n, then 𝔽^{m} (identified with {(a_{1}, a_{2}, …, a_{m}, 0, …, 0)}) is a subspace of 𝔽^{n}.

3. If m ≤ n, then 𝔽^{m}[x] is a subspace of 𝔽^{n}[x], and a subspace of 𝔽[x].

4. If m ≤ n, then C^{n}(ℝ) and C^{∞}(ℝ) are subspaces of C^{m}(ℝ) .

5. In ℝ^{2}, let U = {(x, y) | y = 3x}. Then U is a subspace.

6. More generally, in ℝ^{n}, fix a_{1}, a_{2}, …, a_{n}, and let U = {(x_{1}, x_{2}, …, x_{n}) | a_{1}x_{1} + a_{2}x_{2} + … + a_{n}x_{n} = 0}. Then U is a subspace.

7. If U_{1} and U_{2} are subspaces, so is U_{1} ∩ U_{2}.

• Examples which are not subspaces:

1. In ℝ^{2}, let U = {(x, y) | x = 0 or y = 0} (i.e., U is the union of the x-axis and the y-axis). Then U is not a subspace, since the two vectors (1, 0) and (0, 1) are in U, but (1, 0) + (0, 1) = (1, 1) is not in U.

2. In ℝ^{2}, let U = {(x, y) | x and y are both rational numbers}. Then U is not a subspace, since (1, 1) is an element of U, but \sqrt{2} ∗ (1, 1) = (\sqrt{2}, \sqrt{2}) is not an element of U.

3. In ℝ^{n}, fix a_{1}, a_{2}, …, a_{n}, and let U = {(x_{1}, x_{2}, …, x_{n}) | a_{1}x_{1} + a_{2}x_{2} + … + a_{n}x_{n} = 2}. Then U is not a subspace, since, for example, 0 is not an element of U.

4. In general, if U_{1} and U_{2} are subspaces, U_{1} ∪ U_{2} is not a subspace (except in special cases, such as when U_{1} ⊂ U_{2}).

• If U_{1} and U_{2} are both subspaces of a vector space V, we can define the subspace which is the sum of the two subspaces by:

U_{1} + U_{2} = {u_{1} + u_{2} | u_{1} ∈ U_{1} and u_{2} ∈ U_{2}}.

• If in addition U_{1} ∩ U_{2} = {0}, then each element u ∈ U_{1} + U_{2} can be written in a unique way as u = u_{1} + u_{2}. In this case, we call the sum of U_{1} and U_{2} a direct sum, and write it U_{1} ⊕ U_{2}.

• These ideas generalize in a straightforward way to U_{1} + ⋯ + U_{n} and U_{1} ⊕ ⋯ ⊕ U_{n} for a finite number of subspaces, and U_{1} + U_{2} + ⋯ and U_{1} ⊕ U_{2} ⊕ ⋯ for countably many subspaces.

Exercises: Subspaces ←

1. Show that each of the examples identified in this section as a subspace actually is a vector subspace.

2. Which of the following are vector subspaces of ℝ^{3}?

(a) {(x_{1}, x_{2}, x_{3}) ∈ ℝ^{3} | 3x_{1} + 2x_{2} − x_{3} = 0}

(b) {(x_{1}, x_{2}, x_{3}) ∈ ℝ^{3} | 3x_{1} + 2x_{2} − x_{3} = 4}

(c) {(x_{1}, x_{2}, x_{3}) ∈ ℝ^{3} | x_{1}x_{2}x_{3} = 0}

3. Suppose that U is a vector subspace of V, and V is a vector subspace of W. Show that U is a vector subspace of W.

4. Show that the intersection of any collection of vector subspaces of V is a vector subspace of V.

5. Let

l^{2} = {(a_{i}) ∈ ℝ^{∞} | ∞ ; ∑ ; i=0 |a_{i}|^{2} < ∞}.

Show that l^{2} is a vector subspace of ℝ^{∞}.

6. Let

L^{2} = {f ∈ C^{0}(ℝ) | ∫_{−∞}^{∞} |f(x)|^{2}dx < ∞}. Show that L^{2} is a vector subspace of C^{0}(ℝ).

Linear dependence and independence ←

From here on, we’ll assume that U and V are vector spaces over a field 𝔽.

• Suppose that v_{1}, v_{2}, …, v_{n} are vectors in V, and that for some a_{2}, a_{3}, …, a_{n} ∈ 𝔽,

v_{1} = a_{2}v_{2} + a_{3}v_{3} + … + a_{n}v_{n}.

Then we say that v_{1} is linearly dependent on v_{2}, …, v_{n}. Note that if we move v_{1} to the other side, and let a_{1} = −1, we have a_{1}v_{1} + a_{2}v_{2} + … + a_{n}v_{n} = 0, and not all of the a_{i} are zero (in particular, a_{1} ≠ 0).

This motivates a general definition: a set S of vectors in a vector space V is called linearly dependent if, for some n > 0, and distinct v_{1}, v_{2}, … v_{n} ∈ S, there exist a_{1}, a_{2}, …, a_{n} ∈ 𝔽, not all 0, with

a_{1}v_{1} + a_{2}v_{2} + … + a_{n}v_{n} = 0.

• The flip side of this definition is the following: a set S of vectors in a vector space V is called linearly independent if, given distinct v_{1}, v_{2}, … v_{n} ∈ S, the only way for a_{1}, a_{2}, …, a_{n} ∈ 𝔽 to give

a_{1}v_{1} + a_{2}v_{2} + … + a_{n}v_{n} = 0

is if every one of the a_{i} are zero.

A useful way to think about this is that a set S of vectors is linearly independent if no individual one of the vectors is linearly dependent on a finite number of the rest of them.

• Some examples:

1. In ℝ^{2}, the sets of vectors {(1, 0), (0, 1)}, {(1, 2), (1, 3)}, {(1, 0)} and {(2, 2), (3, 2)} are each linearly independent sets.

2. In ℝ^{2}, none of the sets of vectors {(1, 0), (2, 0)}, {(1, 0), (0, 1), (1, 1)}, {(0, 0), (0, 1)}, {(0, 0)}, {(1, 2), (2, 3), (3, 4), (4, 1)}, nor {(1, 2), (3, 4), (2, 1)} is a linearly independent set.

3. In any vector space, if a set of vectors contains the 0 vector, the set is not linearly independent.

4. In C^{0}(ℝ), if f_{1}(1) = f_{2}(2) = … = f_{n}(n) = 1, but f_{i}(j) = 0 for i ≠ j, then the set of functions {f_{1}, f_{2}, …, f_{n}} is linearly independent.

5. In C^{∞}(ℝ), the set of functions {cos (nx) | n = 0, 1, 2, … } is a linearly independent set.

6. The empty set, {}, is a linearly independent set.

• Mathematics is an extremely precise language. These two sentences do not mean the same thing:

1. The set {v_{1}, v_{2}, …, v_{n}} of vectors is linearly independent.

2. {v_{1}, v_{2}, …, v_{n}} is a set of linearly independent vectors.

One of the hardest parts of doing mathematics is developing your mathematical intuition. It is tempting to imagine that intuition is what you have before you know anything, but that is nonsense. Intuition is just the automatic part of your knowledge, derived from your past experience. Becoming better at mathematics involves learning new mathematics, and then integrating that new knowledge into your intuition. Doing that takes care, precision, and lots of practice!

Exercises: Linear dependence and

independence ←

1. Verify the statements in each of the six examples in this section.

2. Suppose that P_{0}, P_{1}, …, P_{n} are polynomials in 𝔽^{n}[x], and P_{i}(1) = 0 for i = 0, 1, …, n. Show that {P_{0}, P_{1}, …, P_{n}} is not linearly independent in 𝔽^{n}[x].

3. In C^{∞}(ℝ), show that each of these sets of functions is linearly independent:

(a) {x^{n} | n = 0, 1, 2, … }.

(b) {sin(nx) | n = 0, 1, 2, … }.

(c) {e^{nx} | n = 0, 1, 2, … }.

Span of a set of vectors←

• If S is a set of vectors in the vector space V over 𝔽, we define the span of the set S by:

span (S) = {a_{1}v_{1} + a_{2}v_{2} + … + a_{n}v_{n} |

a_{i} ∈ 𝔽, v_{i} ∈ S, n > 0}.

This says that span (S) is the set of all vectors that can be written as finite linear combinations of the vectors in S.

• Examples:

1. In ℝ^{1}, span ({(1)}) is all of ℝ^{1}.

2. In ℝ^{3}, span ({(1, 0, 0), (0, 1, 0)}) is the x − y plane. Similarly, span ({(1, 0, 0), (0, 0, 1)}) is the x − z plane, span ({(0, 1, 0), (0, 0, 1)}) is the y − z plane, and span ({(1, 0, 0), (0, 1, 0), (0, 0, 1)}) is all of ℝ^{3}.

3. Still in ℝ^{3}, span ({(1, 2, 0), (2, 1, 0)}) is the x − y plane, span ({(1, 0, 0)}) is the x axis, span ({(1, 0, 0), (1, 1, 0), (1, 1, 1)}) is all of ℝ^{3}, span ({(1, 2, 3), (1, 1, 0), (1, 1, 1), (2, 1, 4)}) is all of ℝ^{3}, and span ({(1, 2, 0), (1, 1, 0), (3, 1, 0), (2, 1, 0)}) is the x − y plane.

4. In 𝔽^{n}[x], span ({1, x^{1}, x^{2}, …, x^{n}}) = 𝔽^{n}[x].

5. In 𝔽[x], span ({1, x^{1}, x^{2}, … }) = 𝔽[x].

6. In C^{0}(ℝ), span ({f_{1}(x) = 1, f_{2}(x) = x}) is the set of all linear functions, y = mx + b.

7. For any set S, span (S) is a subspace.

8. In particular, span (S) is sometimes defined to be the smallest subspace of V containing S, or the intersection of all subspaces of V that contain S.

Exercises: Span of a set of vectors ←

1. Verify the statements in each of the eight examples in this section.

2. Show that if span ({v_{0}, v_{1}, …, v_{n}}) = V, then span ({v_{0} − v_{1}, v_{1} − v_{2}, …, v_{n−1} − v_{n}, v_{n}}) = V.

Basis for a vector space←

• Suppose S is an ordered set (or list) of vectors in a vector space V. Suppose further that:

1. S is linearly independent, and

2. span (S) = V.

Then we call S a basis for the vector space V.

• Given a basis S for the vector space V, every vector v ∈ V, v ≠ 0, can be written in a unique way as:

v = a_{1}v_{1} + a_{2}v_{2} + … + a_{n}v_{n}

with a_{1}, a_{2}, …, a_{n} ∈ 𝔽, a_{i} ≠ 0, and v_{1}, v_{2}, …, v_{n} ∈ S. For uniqueness, we take the v_{i} to be distinct, and to be in the order they appear in S. We are writing each v as a finite linear combination of the basis vectors.

• Examples:

1. In 𝔽^{3}, the ordered set S = ((1, 0, 0), (0, 1, 0), (0, 0, 1)) is a basis (often called the standard basis). This generalizes in the obvious way to 𝔽^{n}.

2. In 𝔽^{3}, the ordered set S = ((1, 0, 0), (1, 1, 0), (1, 1, 1)) is a basis. So is ((1, 2, 3), (2, 1, 3), (1, 2, 2)).

3. In general, in 𝔽^{n} any linearly independent ordered set S = (v_{1}, v_{2}, …, v_{n}) of size n is a basis.

4. (1, x, x^{2}, x^{3}, …) is a basis for 𝔽[x].

5. If S is a linearly independent ordered set, then S is a basis for span (S).

• If a vector space V has a finite basis S = (v_{1}, v_{2}, …, v_{n}), we say that V is finite dimensional, and we define the dimension of V by:

dim (V) = n.

We define dim ({0}) = 0.

Note that if a vector space V has a finite basis of size n, then every basis for V contains n vectors, and thus the definition makes sense.

For example, dim (𝔽^{n}) = n for any field 𝔽 and n > 0.

We also have that dim (𝔽^{n}[x]) = n + 1.

• If a vector space V has a finite or countably infinite basis S, then we can uniquely represent each vector v with respect to S by a list of the form: v = (b_{1}, b_{2}, …, b_{n}) in the finite case, or v = (b_{1}, b_{2}, …) in the infinite case.

Each b_{i} is either the non-zero coefficient corresponding with the ith element of S from the unique representation described above, or 0 if the basis element does not appear there. In the infinite case, only finitely many of the b_{i} are non-zero.

We represent 0 by (0, 0, …, 0) in the finite case, and by (0, 0, …) in the infinite case.

• In these cases, we also have, for v_{i} ∈ S, that

V = span (v_{1}) ⊕ ⋯ ⊕ span (v_{n})

in the finite case, and

V = span (v_{1}) ⊕ span (v_{2}) ⊕ ⋯

in the countably infinite case.

Exercises: Basis for a vector space ←

1. Verify the statements in each of the five examples in this section.

2. Let U be the subspace of ℝ^{6} given by U = {(x_{1}, x_{2}, x_{3}, x_{4}, x_{5}, x_{6}) ∈ ℝ^{6} | 2x_{1} + 3x_{3} = 0 and x_{2} = 4x_{5}}.

Find a basis for U.

3. Show that if U_{1} and U_{2} are subspaces of a finite dimensional vector space, then dim (U_{1} + U_{2}) = dim (U_{1}) + dim (U_{2}) − dim (U_{1} ∩ U_{2}).

Linear transformations ←

• If U and V are vector spaces over 𝔽, then a function T : U → V is called a linear transformation or linear mapping if

T (a_{1}u_{1} + a_{2}u_{2}) = a_{1}T (u_{1}) + a_{2}T (u_{2})

for all a_{1}, a_{2} ∈ 𝔽 and u_{1}, u_{2} ∈ U.

An equivalent pair of conditions is that T (u_{1} + u_{2}) = T (u_{1}) + T (u_{2}), and T (au) = aT (u).

• For a linear transformation T : U → V, we call U the domain and V the codomain (or sometimes range) of T. We also define the kernel (or null space) of T by:

ker (T) = {u ∈ U | T (u) = 0}.

Further, we define the image (or sometimes range – be careful here!) of T by:

im(T) = {v ∈ V | v = T (u) for some u ∈ U}.

• For a linear transformation T : U → V we have the nice properties:

1. ker (T) is a subspace of U, and im(T) is a subspace of V. (ex)

2. If U is finite dimensional, then

dim (U) = dim (ker (T)) + dim (im(T)).

(ex)

3. If U is finite dimensional with basis (u_{1}, u_{2}, …, u_{n}), and V is finite dimensional with basis (v_{1}, v_{2}, …, v_{m}), then T is determined by its effect on the basis elements u_{i}. There exist a_{ij}, 1 ≤ i ≤ m, 1 ≤ j ≤ n with:

T (u_{j}) = a_{1j}v_{1} + a_{2j}v_{2} + … + a_{mj}v_{m},

and in general, if u = ∑_{j} b_{j}u_{j}, then

T (u) = m ; ∑ ; i=1 n ; ∑ ; j=1 a_{ij}b_{j}v_{i}.

4. Thus, given particular bases for U and V, we can represent the linear transformation T by the matrix

[T] = [ \begin{matrix}a_{11} & a_{12} & … & a_{1n} \\ a_{21} & a_{22} & … & a_{2n} \\ ⋮ & ⋮ & ⋮ & ⋮ \\ a_{m1} & a_{m2} & … & a_{mn} \end{matrix} ]

or [T]_{ij} = a_{ij}.

5. If we represent the vector u ∈ U by the column matrix [b_{1}, b_{2}, …, b_{n}]^{t}, then we have

[T (u)] = [T][u]
= [ \begin{matrix}a_{11} & a_{12} & … & a_{1n} \\ a_{21} & a_{22} & … & a_{2n} \\ ⋮ & ⋮ & ⋮ & ⋮ \\ a_{m1} & a_{m2} & … & a_{mn} \end{matrix} ] [ b_{1} ; b_{2} ; ⋮ ; b_{n} ]

= [ a_{11} b_{1} + a_{12} b_{2} + … + a_{1n}b_{n} ; a_{21} b_{1} + a_{22} b_{2} + … + a_{2n}b_{n} ; ⋮ ; a_{m1}b_{1} + a_{m2}b_{2} + … + a_{mn}b_{n}  ]

6. Sometimes it is more convenient to use the Einstein summation conventions, where we would write:

T (u)_{i} = T_{i}^{j}u_{j}

with implied summation over the repeated upper and lower indices.

7. If T_{1} : U → V and T_{2} : U → V are two linear transformations, we can define the sum of the two transformations as

(T_{1} + T_{2})(u) = T_{1}(u) + T_{2}(u),

and we can define scalar multiplication of a linear transformation T by a as

(a ∗ T)(u) = a ∗ (T (u)).

We can thus define L(U, V) to be the space of all linear transformations from U to V. We define the zero transformation 0 : U → V by 0(u) = 0. With these definitions, L(U, V) is a vector space. (ex)

8. Given particular bases for U and V, the matrix representation of T_{1} + T_{2} is given by: [T_{1} + T_{2}] = [T_{1}] + [T_{2}].

9. If U and V are finite dimensional, then L(U, V) is also finite dimensional, and

dim (L(U, V)) = dim (U) ∗ dim (V).

(ex)

10. If T : U → V and S : V → W are linear transformations, then the composition S ∘ T : U → W is also a linear transformation. (ex) Recall that (S ∘ T)(u) = S(T (u)). Given particular bases for U, V, and W, the matrix representation of S ∘ T is the matrix product [S ∘ T] = [S][T]. We usually abbreviate S ∘ T as ST. It is worth noting that unless W ⊆ U, it doesn’t even make sense to talk about T ∘ S.

11. The matrix of the composition of two linear transformations is given by the product of the two matrices, given by:

[ST] = [S][T] =

[ \begin{matrix}a_{11} & a_{12} & … & a_{1n} \\ a_{21} & a_{22} & … & a_{2n} \\ ⋮ & ⋮ & ⋮ & ⋮ \\ a_{m1} & a_{m2} & … & a_{mn} \end{matrix} ] [ \begin{matrix}b_{11} & b_{12} & … & b_{1p} \\ b_{21} & b_{22} & … & b_{2p} \\ ⋮ & ⋮ & ⋮ & ⋮ \\ b_{n1} & b_{n2} & … & b_{np} \end{matrix} ]

=

[ \begin{matrix}∑_{i=1}^{n} a_{1i}b_{i1} & ∑_{i=1}^{n} a_{1i}b_{i2} & … & ∑_{i=1}^{n} a_{1i}b_{ip} \\ ∑_{i=1}^{n} a_{2i}b_{i1} & ∑_{i=1}^{n} a_{2i}b_{i2} & … & ∑_{i=1}^{n} a_{2i}b_{ip} \\ ⋮ & ⋮ & ⋮ & ⋮ \\ ∑_{i=1}^{n} a_{mi}b_{i1} & ∑_{i=1}^{n} a_{mi}b_{i2} & … & ∑_{i=1}^{n} a_{mi}b_{ip} \end{matrix} ]

12. We also have the distributive property:

S(T_{1} + T_{2}) = (ST_{1}) + (ST_{2}).

The matrix representation for this is:

[S][T_{1} + T_{2}] = [S][T_{1}] + [S][T_{2}].

• Some examples of linear transformations:

1. The function T : ℝ^{2} → ℝ given by T ((x, y)) = x + y is linear.

2. The function T : ℝ^{2} → ℝ^{3} given by T ((x, y)) = (x + 2y, x − 2y, x − y) is linear.

3. Given a_{1}, a_{2}, …, a_{n} ∈ 𝔽, and V a finite dimensional vector space over 𝔽 with basis (v_{1}, v_{2}, …, v_{n}), the function T : V → 𝔽 given by T (b_{1}v_{1} + … + b_{n}v_{n}) = a_{1}b_{1} + … + a_{n}b_{n} is linear. In general, for a vector space V, a linear transformation T : V → 𝔽 is called a linear functional. The study of these transformations is called functional analysis.

4. The function D : 𝔽^{n}[x] → 𝔽^{n−1}[x] given by D(a_{n}x^{n} + … + a_{1}x + a_{0})

= na_{n}x^{n−1} + … + 2a_{2}x + a_{1} is linear. This linear transformation is called the derivative …

5. Similarly, we have the derivative transformation D : 𝔽^{∞}[x] → 𝔽^{∞}[x] given by

D( ∞ ; ∑ ; n=0 a_{n}x^{n}) = ∞ ; ∑ ; n=1 n ∗ a_{n}x^{n−1}.

6. The shift map S : 𝔽^{∞} → 𝔽^{∞} is given by: S((a_{0}, a_{1}, …)) = (a_{1}, a_{2}, …).

7. The difference map Δ : 𝔽^{∞} → 𝔽^{∞} is given by:

Δ((a_{0}, a_{1}, …)) = (a_{1} − a_{0}, a_{2} − a_{1}, …),

or, abbreviating (a_{0}, a_{1}, …) by (a_{n}),

Δ((a_{n})) = (a_{n+1} − a_{n}).

Exercises: Linear transformations ←

1. Verify each of the statements marked (ex) in this section.

2. Verify that each of the 7 examples actually are linear transformations.

3. Show that the usual calculus derivative \frac{d}{dx} : C^{∞}(ℝ) → C^{∞}(ℝ) given by:

\frac{d f(x)}{dx} = lim_{h→0} \frac{f(x + h) − f(x)}{h}

is a linear transformation.

4. Is the usual calculus indefinite integral

∫ f(x)dx

a linear transformation? Why or why not? What about the definite integral?

Morphisms – mono, epi, and iso ←

• In algebra, we call a function that preserves structure a morphism. In our current context, a linear transformation preserves the linear structure of a vector space in the sense that T (u_{1} + u_{2}) = T (u_{1}) + T (u_{2}) and T (au) = aT (u). Thus, linear transformations are morphisms in the category of vector spaces.

• We will be particularly interested in morphisms that have additional properties. Specifically, we are likely to look for morphisms T : U → V which have one or more of the properties:

1. One-to-one. Such transformations are also called injective, or monomorphisms. These transformations have the property that if u_{1} ≠ u_{2}, then T (u_{1}) ≠ T (u_{2}). This says that different things get sent different places – that is, no two things get sent to the same place. Monomorphisms are nice because the subspace im(T) ⊂ V looks just like U.

2. Onto. Such transformations are also called surjective, or epimorphisms. These transformations have the property that for every v ∈ V, there exists some u ∈ U with T (u) = v. This says that every element of V is hit by some element of U. Another way to say this is that im(T) = V. Epimorphisms are nice, because the algebraic properties of V will be reflected back in U.

3. Both one-to-one and onto. Such transformations are also called bijective. A bijective function f : X → Y has an inverse f^{−1} : Y → X with the properties that for all x ∈ X and all y ∈ Y, f^{−1}(f(x)) = x and f(f^{−1}(y)) = y. A bijective morphism whose inverse also preserves algebraic structure is called an isomorphism. In linear algebra, we have the nice property that if a linear transformation is bijective, then its inverse is also linear, and thus it is an isomorphism.

4. If there is an isomorphism T : U → V, we say that the spaces U and V are isomorphic. If two spaces are isomorphic, then they share all relevant properties. Thus, two isomorphic vector spaces are indistinguishable as vector spaces except for a renaming of the elements.

• We have the nice fact that a linear transformation T : U → V is a monomorphism (is one-to-one) if and only if ker (T) = {0}.

Pf.: Suppose T is a monomorphism. We know that for every linear transformation, T (0) = 0. Then, since T is a monomorphism, we know that if T (u) = 0 = T (0), it must be that u = 0. Thus ker (T) = {0}.

On the other hand, suppose that ker (T) = {0}. Then, if T (u_{1}) = T (u_{2}), we will have 0 = T (u_{1}) − T (u_{2}) = T (u_{1} − u_{2}). But this means that u_{1} − u_{2} ∈ ker (T) and hence, since we are assuming that ker (T) = {0}, we must have u_{1} − u_{2} = 0, or u_{1} = u_{2}. By the contrapositive, this means that if u_{1} ≠ u_{2}, then T (u_{1}) ≠ T (u_{2}). Q.E.D.

(I had to do at least one proof, didn’t I? :-)

• An important example of an isomorphism is the identity transformation I_{U} : U → U given by I_{U}(u) = u. (ex)

• If T : U → V is a monomorphism, then there is a linear transformation S_{1} : im(T) → U with S_{1}(T (u)) = (S_{1}T)(u) = u for all u ∈ U. This says that S_{1}T = I_{U}. S_{1} is called a left (partial) inverse of T. (ex)

• If T : U → V is an epimorphism, then there is a linear transformation S_{2} : V → U with T (S_{2}(v)) = (T S_{2})(v) = v for all v ∈ V. This says that T S_{2} = I_{V}. S_{2} is called a right (partial) inverse of T. (ex)

• If T : U → V is an isomorphism, then since T is an epimorphism, both S_{1} and S_{2} (as above) exist. Also, im(T) = V. We therefore have that for all v ∈ V,

S_{1}(v) = S_{1}((T S_{2})(v)) = (S_{1}T)(S_{2}(v)) = S_{2}(v).

Thus S_{1} = S_{2}. In this case, the (two-sided) inverse of T exists, and we have T^{−1} = S_{1} = S_{2}.

• If U and V are finite dimensional vector spaces over 𝔽 with dim (U) = dim (V), then U and V are isomorphic. (ex)

(Big) hint for proof: Let (u_{1}, u_{2}, …, u_{n}) and (v_{1}, v_{2}, …, v_{n}) be bases for U and V respectively. Define T : U → V by T (u_{i}) = v_{i} for 1 ≤ i ≤ n, and extend by linearity. Make sense of the phrase “extend by linearity,” and then show that T is an isomorphism.

• We also have the nice fact that if dim (U) = dim (V) = n, U, V over 𝔽, and T : U → V is linear, then the following are equivalent: (ex)

1. T is a monomorphism.

2. T is an epimorphism.

3. T is an isomorphism. This means, for example, that such a T is onto if and only ker (T) = 0.

• If T : U → V and S : V → W, then (ex)

1. If both S and T are monomorphisms, then so is ST.

2. If ST is a monomorphism, then so is T.

3. If both S and T are epimorphisms, then so is ST.

4. If ST is an epimorphism, then so is S.

• If T : V → V and S : V → V, and ST = T S, then (ex)

1. Both S and T are monomorphisms if and only if ST is a monomorphism.

2. Both S and T are epimorphisms if and only if ST is an epimorphism.

• Let matrix (𝔽, n, m) be the space of all n by m matrices with entries from 𝔽. We use ordinary entry by entry matrix addition, where (A + B)_{ij} = A_{ij} + B_{ij}, scalar multiplication, where (aA)_{ij} = aA_{ij}, and let (0)_{ij} = 0. Then matrix (𝔽, n, m) is an n ∗ m-dimensional vector space over 𝔽. If dim (U) = m and dim (V) = n, then we can define

mat : L(U, V) → matrix (𝔽, n, m)

by mat (T) = [T].

This transformation is an isomorphism. (ex)

Exercises: Morphisms – mono, epi, and iso ←

1. Verify each of the statements marked (ex) in this section.

2. Show that if a function has an inverse, then the inverse is unique.

3. Consider the function T : 𝔽^{n}[x] → 𝔽^{n+1} given by

T (a_{0} + a_{1}x + … + a_{n}x^{n}) = (a_{0}, …, a_{n}).

Show that this function is an isomorphism.

4. Consider the function T : ℂ → matrix (ℝ, 2, 2) given by

T (a + bi) = [ \begin{matrix}a & −b \\ b & a \end{matrix} ]

(a) Show that if we consider ℂ as a real vector space, this function is a monomorphism.

(b) Show that this function also respects complex multiplication, that is,

T ((a + bi)(c + di)) = T (a + bi)T (c + di).

5. More generally, consider the function T : matrix (ℂ, n, n) → matrix (ℝ, 2n, 2n) given by

T ([ \begin{matrix}a_{11} + b_{11} i & ⋯ & a_{1n} + b_{1n}i \\ ⋮ & & ⋮ \\ a_{n1} + b_{n1}i & ⋯ & a_{nn} + b_{nn}i \end{matrix} ])

= [ \begin{matrix}a_{11} & −b_{11} & ⋯ & a_{1n} & −b_{1n} \\ b_{11} & a_{11} & ⋯ & b_{1n} & a_{1n} \\ ⋮ & ⋮ & & ⋮ & ⋮ \\ a_{n1} & −b_{n1} & ⋯ & a_{nn} & −b_{nn} \\ b_{n1} & a_{n1} & ⋯ & b_{nn} & a_{nn} \end{matrix} ]

(a) Show that if we consider matrix (ℂ, n, n) as a real vector space, this function is a monomorphism.

(b) Show that this function also respects matrix multiplication, that is,

T (AB) = T (A)T (B).

6. What the heck. Let ℍ be the quaternions (described above). Consider the function T : matrix (ℍ, n, n) → matrix (ℂ, 2n, 2n) given by

T ([ \begin{matrix}α_{11} + β_{11} j & ⋯ & α_{1n} + β_{1n}j \\ ⋮ & & ⋮ \\ α_{n1} + β_{n1}j & ⋯ & α_{nn} + β_{nn}j \end{matrix} ])

= [ \begin{matrix}α_{11} & −\overline{β_{11}} & ⋯ & α_{1n} & −\overline{β_{1n}} \\ β_{11} & α_{11} & ⋯ & β_{1n} & α_{1n} \\ ⋮ & ⋮ & & ⋮ & ⋮ \\ α_{n1} & −\overline{β_{n1}} & ⋯ & α_{nn} & −\overline{β_{nn}} \\ β_{n1} & α_{n1} & ⋯ & β_{nn} & α_{nn} \end{matrix} ]

(a) Show that if we consider matrix (ℍ, n, n) as a complex vector space, this function is a monomorphism.

(b) Show that this function also respects matrix multiplication, that is,

T (AB) = T (A)T (B).

Thus, if we denote by O(n), U(n), and Sp(n) the distance preserving linear operators on ℝ^{n}, ℂ^{n}, and ℍ^{n} respectively (called the orthogonal, unitary, and symplectic groups), then we have the monomorphisms:

⋯ → O(n) → Sp(n) → U(2n) → O(4n)

→ Sp(4n) → U(8n) → O(16n) → ⋯

Linear operators ←

• If T : V → V is a linear transformation, we call T a linear operator on V. Note that if S and T are operators on V, then so is ST. We can abbreviate L(V, V) as L(V).

L(V) has the algebraic structure of a ring with identity. A ring is similar to a field (as defined above), except without the requirements that multiplication be commutative and that there be multiplicative inverses for non-zero elements. The identity element is I_{V}. L(V) is a non-commutative ring, since in general ST ≠ T S. This is reflected in the fact that matrix multiplication is non-commutative. Only in very special cases is it true that [S][T] = [T][S] (for example, if both [S] and [T] are diagonal matrices, with [S]_{ij} = 0 for i ≠ j, and [T]_{ij} = 0 for i ≠ j).

• Note that for operators, it makes sense to talk about T ∘ T, and we can thus define T^{n} = T ∘ T ∘ … ∘ T (n times). We also define T^{0} = I, the identity operator.

• Thus, if we have a polynomial P (x) = a_{n}x^{n} + a_{n−1}x^{n−1} + … + a_{1}x + a_{0} from 𝔽[x], we can talk about the polynomial in T:

P (T) = a_{n}T^{n} + a_{n−1}T^{n−1} + … + a_{1}T + a_{0}. (The a_{0} term stands for a_{0}I.) This is an operator on V which acts on vectors as:

P (T)(v) = a_{n}T^{n}(v) + … + a_{1}T (v) + a_{0}v.

• In fact, we can generalize this to power series. If

p(x) = ∞ ; ∑ ; n=0 a_{n}x^{n},

then we can define

p(T) = ∞ ; ∑ ; n=0 a_{n}T^{n}.

• This means, for example, that we can define the exponential of an operator:

exp (T) = ∞ ; ∑ ; n=0 \frac{1}{n!}T^{n}.

• We can even define the cosine or sine of an operator, etc.:

cos (T) = ∞ ; ∑ ; n=0 \frac{(−1)^{n}}{(2n)!} T^{2n},

sin(T) = ∞ ; ∑ ; n=0 \frac{(−1)^{n}}{(2n + 1)!} T^{2n+1} .

• We can talk about square roots of an operator, saying that S is a square root of T if S^{2} = T. It is unlikely that S is unique.

• We can also talk about logarithms of an operator, saying that S is a logarithm of T if exp (S) = T. Again, it is unlikely that S is unique.

• Examples of linear operators:

1. A first important example begins like this: suppose we have a system of m linear equations in n variables

a_{11} x_{1} + a_{12} x_{2} + … + a_{1n}x_{n}  = b_{1}
a_{21} x_{1} + a_{22} x_{2} + … + a_{2n}x_{n}  = 2_{1}
⋮ ⋮
a_{m1}x_{1} + a_{m2}x_{2} + … + a_{mn}x_{n}  = b_{m}.

If we let A : 𝔽^{n} → 𝔽^{m} be the linear transformation with [A]_{ij} = a_{ij}, x be the vector (x_{1}, x_{2}, …, x_{n})^{t}, and b the vector (b_{1}, b_{2}, …, b_{m})^{t}, then we can rewrite the equation in the form:

Ax = b.

We then have several possibilities.

(a) The first case is when m = n. There are then two possibilities:

i. If A is an isomorphism, then A^{−1} exists, and we can solve the equation for x:

x = A^{−1}Ax = A^{−1}b.

ii. If A is not an isomorphism, then in particular A is not a an epimorphism, and thus it is possible that b ∉ im(A). In this case, there are no solutions to the equation. On the other hand, if b ∈ im(A) there is at least one solution x_{0} with Ax_{0} = b. Note, though, that we also know A is not a monomorphism, and hence dim (ker (A)) ≥ 1. Then, if z ∈ ker (A), we have A(x_{0} + z) = Ax_{0} + Az = Ax_{0} + 0 = Ax_{0} = b, and so x_{0} + z is another solution.

Furthermore, if y is another solution with Ay = b, then A(x_{0} − y) = Ax_{0} − Ay = b − b = 0. This means x_{0} − y ∈ ker (A), and so y = x_{0} + z for some z ∈ ker (A). Thus, if x_{0} is a particular solution, then every solution is of the form x_{0} + z for some z ∈ ker (A). The space of solutions is then a translation of the kernel of A, of the form x_{0} + ker (A). We then only need to find one particular solution. In this case, we have broken the problem down into two parts: first, we solve Ax = 0 (called the homogeneous equation), then we find a single solution Ax_{0} = b. For 𝔽 = ℝ or ℂ, there will be infinitely many solutions.

(b) The second case is when m < n. In this case, A cannot be a monomorphism, and things are then similar to the second part of the previous case. If A is an epimorphism, there is sure to be at least one solution. Again, we solve the homogeneous case, and then find one particular solution. If A is not an epimorphism, there may not be any solutions. Otherwise, as above, in general we will have infinitely many solutions.

(c) If m > n, then A cannot be an epimorphism, and hence there may not be any solutions. If A is a monomorphism, if there is a solution, there will be just one. If A is not a monomorphism, if there is one solution, there will be infinitely many, as above.

(d) In all three cases, we can start by trying to find the inverse (or left partial inverse) of A. If A is a monomorphism, we can expect to be successful, and to be able to find the unique solution if it exists. If A is not a monomorphism, we solve the homogeneous equation, and then look for a single particular solution.

(e) Question: How can we tell if A is a monomorphism? How can we find A^{−1}, or at least S_{1}, the left partial inverse of A?

2. A second important example is the derivative operator

D : C^{∞}(ℝ) → C^{∞}(ℝ).

Note that this is a linear operator, since D(f + g) = D(f) + D(g), and D(af) = aD(f). Note that ker (D) = span ({1}), the one-dimensional space consisting of all constant functions. If we collapse ker (D) down to nothing (in technical terms, form the quotient space …) then we can think of D as an isomorphism (on the quotient space). D has an inverse,

∫ : C^{∞}(ℝ) → C^{∞}(ℝ).

We then have (a variant of) the fundamental theorem of calculus:

D ∫ (f) = ∫ D(f) = f

(on the reduced space – that is, up to a constant).

3. We can then consider the general linear ordinary differential operators with constant coefficients. These are operators in ℝ[D], that is operators of the form:

P (D) = a_{n}D^{n} + … + a_{1}D + a_{0}.

These operators act on a function f ∈ C^{∞}(ℝ) as

P (D)(f) = a_{n}D^{n}(f)+…+a_{1}D(f)+a_{0}f.

We then have the general nth order linear ordinary differential equation with constant coefficients:

P (D)(f) = g.

We can work on solving these equations with an approach similar to the method for systems of linear equations.

We first note that if the function f_{0} is a solution to this equation, and z = z(x) ∈ ker (P (D)), then f_{0} + z is also a solution, and if f_{1} is another solution, then P (D)(f_{0} − f_{1}) = P (D)(f_{0}) − P (D)(f_{1}) = g − g = 0. Thus, all solutions are of the form f_{0} + z where f_{0} is some particular solution, and z ∈ ker (P (D)). We thus separate the problem into two parts. First we solve the associated homogeneous equation:

P (D)(f) = 0

to find ker (P (D)), and then we look for a single particular solution to the original equation. In general, we have that dim (ker (P (D))) = n, the degree of P. This is not an entirely obvious fact, but it is not counterintuitive … Hence what we need to do is find n functions which form a basis for ker (P (D)). What we need, then, are n linearly independent functions each of which is a solution to the homogeneous equation. In theory (:-) this is not too hard. We note first that for the first-order case, we have the solution:

(D − r)(e^{rx}) = D(e^{rx}) − re^{rx}
= re^{rx} − re^{rx}
= 0,

and, in the kth order extension of this,

(D − r)^{k}(x^{k−1}e^{rx}) = 0.

We also have that the set of functions A = {x^{j}e^{r_{i}x} | 0 ≤ j ≤ k, 1 ≤ i ≤ m} is linearly independent. From this we see how to solve equations of the form:

_{i}^{∏}(D − r_{i})^{k_{i}}(f) = 0.

Now, consider the operator

D^{2} − 2sD + s^{2} + t^{2}

for s, t ∈ R. We have that (D^{2}−2sD+s^{2}+t^{2})^{k}(x^{k−1}e^{sx} cos (tx)) = 0, (D^{2}−2sD+s^{2}+t^{2})^{k}(x^{k−1}e^{sx} sin(tx)) = 0. We can think of this as α = s + ti, and that we are working with D^{2} − 2sD + s^{2} + t^{2} = (D − α)(D − ᾱ). We have that the set of functions

B = {x^{j}e^{s_{k}x} cos (t_{k}x), x^{m}e^{s_{n}x} sin(t_{n}x)} is also linearly independent. From this we see how to solve equations of the form:

_{i}^{∏}(D^{2} − 2s_{i}D + s_{i}^{2} + t_{i}^{2})^{k_{i}}(f) = 0.

We can now put together all the pieces to solve the homogeneous equation P (D) = 0. We use the fact that any polynomial over ℝ can be completely factored as

P (D) = _{i}^{∏}(D−r_{i})^{k_{i}} _{j}^{∏}(D−α_{j})^{k_{j}}(D−ᾱ_{j})^{k_{j}}

with r_{i} ∈ R and α_{j} ∈ C. To solve the inhomogeneous equation, we need only to find one particular solution of P (D)(f) = g. This is just the bare beginnings of techniques for solving differential equations, but it gives the flavor of some relatively powerful methods, and the role that linear algebra plays. I haven’t even mentioned the issues of initial values/boundary conditions. For much more on these topics, look at a book such as Elementary Differential Equations with Linear Algebra, by Finney and Ostberg.

4. These approaches generalize to linear ordinary differential operators with non-constant coefficients:

a_{n}(x)D^{n} + a_{n−1}(x)D^{n−1} + ⋯ + a_{0}(x),

to systems of linear ordinary differential operators, and on to linear partial differential operators and systems of linear partial differential operators. But I think for now someone else will have to write that up …:-)

5. We can develop a similar approach to solving linear discrete difference equations. For example, the difference equation (a_{n+2} − a_{n+1} − a_{n}) = (0) has as a solution the Fibonnacci sequence (a_{n}) = (1, 1, 2, 3, 5, 8, …). We could work with the discrete version Δ of the differential operator D, where Δf(x) = (f(x + 1) − f(x))/1 = f(x + 1) − f(x). In the discrete case, we can’t let h go to zero.

We would then have Δ((a_{n})) = (a_{n+1} − a_{n}). Our example difference equation would be

(Δ^{2} + Δ − 1)((a_{n})) = (0).

Often, however, it is more convenient to work with the discrete increment operator E, with E(f(x)) = f(x + 1), or E((a_{n})) = (a_{n+1} ). Both Δ and E are linear operators. Our example equation is then E^{2}((a_{n})) = E((a_{n})) + (a_{n}), or

(E^{2} − E − 1)((a_{n})) = (0).

In a general form, we have a polynomial P (x) for an equation of the form P (E)((a_{n})) = (0). We can then find a general solution by using the facts that

(E − α)^{k}((n^{j}α^{n})) = (0)

for 0 ≤ j < k, and

{(n^{j}α^{n}) | 0 ≤ j < k}

is a linearly independent set, etc.

6. Of course, all of these are the easy cases. The hard ones are the nonlinear equations …

7. At times, this reminds me of a comment made by my Ph.D. thesis advisor, after I had finished the proof of my main result for all odd primes. He said there were three ways to think about it:

(a) I had handled infinitely many cases, and omitted only one, the even prime 2.

(b) I had done half the cases. I had handled all the odd primes, but none of the even ones.

(c) I had dealt with all the uninteresting cases, but not the single interesting case :-)

Exercises: Linear operators ←

1. Find a linear operator S such that

S^{2} = [ \begin{matrix}−1 & 0 \\ 0 & −1 \end{matrix} ]

2. If S is the operator

S = [ \begin{matrix}0 & −\frac{π}{4} \\ \frac{π}{4} & 0 \end{matrix} ] ,

what is exp (S)?

3. Solve the system of equations:

\begin{aligned}3x_{1} & + & x_{2} & − & 2x_{3} & = & 3 \\ x_{1} & − & 2x_{2} & − & x_{3} & = & 1 \\ 2x_{1} & − & 4x_{2} & − & x_{3} & = & 3 \end{aligned}

4. Solve the differential equation:

\frac{d^{5}f}{dx^{5}} − 7\frac{d^{4}f}{dx^{4}} + 23\frac{d^{3}f}{dx^{3}} − 45\frac{d^{2}f}{dx^{2}} + 48\frac{df}{dx} − 20f

= 26 cos (x) − 18 sin(x)

Hint:

x^{5} − 7x^{4} + 23x^{3} − 45x^{2} + 48x − 20

= (x − 1)(x − 2)^{2}(x^{2} − 2x + 5)

5. Find the general solution to the difference equation

(E^{2} − E − 1)((a_{n})) = (0)

6. Find the general solution to the difference equation

(E^{4} + 2E^{2} + 1)((a_{n})) = (0)

7. Verify that (a) D, the derivative, is a linear operator.

(b) (D − r)^{k}(x^{k−1}e^{rx}) = 0

(D^{2} − 2sD + s^{2} + t^{2})^{k}(x^{k−1}e^{sx} cos (tx))  = 0
(D^{2} − 2sD + s^{2} + t^{2})^{k}(x^{k−1}e^{sx} sin(tx))  = 0

(c) E, the discrete increment, is a linear operator.

(d) (E − α)^{k}((n^{j}α^{n})) = (0) for 0 ≤ j < k.

8. What happens in nonlinear cases? Sometimes they are manageable, sometimes not.

(a) Solve the differential equation Df = b ∗ f ∗ (1 − f).

(b) What can be said about the difference equation

E((a_{n})) = (b ∗ a_{n} ∗ (1 − a_{n}))?

Note: this is often called the logistics equation.

Normed linear spaces ←

• A norm on a real or complex vector space

V is a function

|| · || : V → ℝ

with the properties, for v, v_{1}, v_{2} ∈ V,

1. ||v|| ≥ 0

2. ||v|| = 0 if and only if u = 0

3. ||av|| = |a| ||v||

4. ||v_{1} + v_{2}|| ≤ ||v_{1}|| + ||v_{2}||. A space with such an associated function is called a normed linear space.

• Using the norm, we can define a topology on the space, and can then talk about continuity of functions or operators, limits of sequences, etc.

I won’t go into this much here beyond a few examples, but good places to look are books on Hilbert Spaces and/or functional analysis. There are a few books indicated in the references.

• Using the norm, we can define a metric on V by: d(v_{1}, v_{2}) = ||v_{1} − v_{2}||. Metrics have the properties:

1. d(v_{1}, v_{2}) ≥ 0

2. d(v_{1}, v_{2}) = 0 iff v_{1} = v_{2}

3. d(v_{1}, v_{2}) = d(v_{2}, v_{1})

4. d(v_{1}, v_{2}) ≤ d(v_{1}, v_{3}) + d(v_{3}, v_{2}) (the triangle inequality).

• Some examples:

1. On ℝ^{n} or ℂ^{n}, we have the norm

||(a_{1}, a_{2}, …, a_{n})||_{2} = (|a_{1}|^{2}+…+|a_{n}|^{2})^{1/2} .

This is usually called the Euclidean norm. We can also think of this in terms of the inner product given by ⟨(a_{1}, a_{2}, …, a_{n}), (b_{1}, b_{2}, …, b_{n})⟩

= a_{1}b̄_{1} + a_{2}b̄_{2} + ⋯ + a_{n}b̄_{n}. We then have ||v||_{2} = ⟨v, v⟩^{1/2} .

2. We can generalize, for p > 0, to

||(a_{1}, a_{2}, …, a_{n})||_{p} = (|a_{1}|^{p}+…+|a_{n}|^{p})^{1/p},

and

||(a_{1}, a_{2}, …, a_{n})||_{∞} = max (|a_{1}|, …, |a_{n}|).

These are called the p-norms and the ∞-norm (or max-norm).

A fun little exercise is to draw the circle of radius 1 in ℝ^{2} for each of these norms:

circle_{p}(1) = {(x, y) ∈ ℝ^{2} | ||(x, y)||_{p} = 1}

for p > 0, or for p = ∞. One of these constitutes a “proof” that a square is a circle :-)

3. We can generalize this to ℝ^{∞} or ℂ^{∞}:

||(a_{1}, a_{2}, …)||_{p} = ( ∞ ; ∑ ; n=1 |a_{n}|^{p})^{1/p} ,
||(a_{1}, a_{2}, …)||_{∞} = sup_{n} (|a_{n}|).

For this to make sense, we will have to limit ourselves to the subspaces where the sum converges:

l^{p} = {(a_{1}, a_{2}, …) | ( ∞ ; ∑ ; n=1 |a_{n}|^{p})^{1/p} < ∞},
l^{∞} = {(a_{1}, a_{2}, …) | sup_{n} (|a_{n}|) < ∞}.

4. On C^{0}(ℝ) or C^{0}(ℂ), we can define

||f||_{p} = (∫ |f|^{p})^{1/p}

||f||_{∞} = sup_{x} (|f(x)|).

Again, we limit ourselves to the subspaces where these are < ∞, and call the corresponding spaces L^{p} or L^{∞}.

5. We can think each of these Euclidean norms, || · ||_{2}, as coming from an inner product:

||(a_{1}, a_{2}, …)||_{2} = ⟨(a_{1}, …), (a_{1}, …)⟩ ^{1/2}
= ( ∞ ; ∑ ; n=1 a_{n}ā_{n})^{1/2}
||f||_{2} = ⟨f, f⟩^{1/2}
= (∫ ff̄)^{1/2} .

When a complex Euclidean normed linear space is complete (that is, every Cauchy sequence converges), it is called a Hilbert space.

Exercises: Normed linear spaces ←

1. Verify that each of the examples actually are norms.

2. In ℝ^{2}, draw the unit circles (a) circle_{1}(1) = {(x, y) ∈ ℝ^{2} | ||(x, y)||_{1} = 1} (b) circle_{3/2} (1) = {(x, y) ∈ ℝ^{2} |

||(x, y)||_{3/2} = 1}

(c) circle_{2}(1) = {(x, y) ∈ ℝ^{2} | ||(x, y)||_{2} = 1} (d) circle_{3}(1) = {(x, y) ∈ ℝ^{2} | ||(x, y)||_{3} = 1} (e) circle_{∞}(1) = {(x, y) ∈ ℝ^{2} |

||(x, y)||_{∞} = 1} 3. Suppose that V is a real or complex

vector space. An inner product on V is a conjugate-bilinear function on V:

< ·, · >: V xV → 𝔽

where, for all v_{1}, v_{2}, v ∈ V, and α ∈ 𝔽,

< v, v > ≥ 0
< v, v > = 0 iff v = 0
< v_{1} + v_{2}, v > = < v_{1}, v > + < v_{2}, v >
< v_{1}, v_{2} > = \overline{< v_{2}, v_{1} >}
< αv_{1}, v_{2} > = α < v_{1}, v_{2} > .

Show that for an inner product,

< v, v_{1} + v_{2} > = < v, v_{1} > + < v, v_{2} >
< v_{1}, αv_{2} > = \overline{α} < v_{1}, v_{2} > .

Show that for a finite dimensional real or complex vector space V with basis (v_{1}, v_{2}, …, v_{n}), the function f : V xV → 𝔽 given by

f (_{i}^{∑} a_{i}v_{i}, _{i}^{∑} b_{i}v_{i}) = _{i}^{∑} a_{i}\overline{b_{i}}

is an inner product.

4. Recall that a linear functional on a vector space V is a linear map f : V → 𝔽. For a finite dimensional real or complex inner product space V, define the dual space of V to be the space

V^{∗} = {f : V → 𝔽 | f is a linear functional }

Show that V^{∗} is a vector space over 𝔽, and that V and V^{∗} are isomorphic to each other.

Hint: Show that every f ∈ V^{∗} corresponds with a function of the form

< v_{f}, · >: V → 𝔽

for some v_{f} ∈ V.

Eigenvectors and eigenvalues ←

• Suppose T is a linear operator on V. Then λ ∈ 𝔽 is called an eigenvalue or characteristic value of T if there exists v ≠ 0 in V with T (v) = λv. In this case we call v an eigenvector or characteristic vector of T.

• An equivalent definition is that λ is an eigenvalue of T if and only if (T − λI)v = 0 for some v ≠ 0. This follows from the fact that 0 = (T − λI)v = T (v) − λv if and only if T (v) = λv. Note that this also means that ker (T − λI) ≠ {0}. Thus, λ is an eigenvalue if and only if (T − λI) is not a monomorphism. In the finite dimensional case, this is equivalent to (T − λI) not being invertible.

• Now suppose that λ_{1}, …, λ_{k} are distinct eigenvalues of T (i.e., λ_{i} ≠ λ_{j} for i ≠ j), with associated eigenvectors v_{1}, …, v_{k}. Then {v_{1}, …, v_{k}} is a linearly independent set.

• This means in particular that if T ∈ L(V) with dim (V) = n, and T has n distinct eigenvalues λ_{i} with associated eigenvectors v_{i}, then the set S = (v_{1}, …, v_{n}) is a basis for V. Furthermore, the matrix representation of T with respect to the basis S is

[T] = [ \begin{matrix}λ_{1} & 0 & … & 0 \\ 0 & λ_{2} & … & 0 \\ ⋮ & ⋮ & ⋮ & ⋮ \\ 0 & 0 & … & λ_{n} \end{matrix} ]

or [T]_{ii} = λ_{i}, [T]_{ij} = 0 for i ≠ j. This is also sometimes written as

[T] = diag (λ_{1}, …, λ_{n}).

Eigenvalues and eigenvectors can thus give us a very simple representation for T.

• Not all linear operators have eigenvalues (or eigenvectors). For example, the linear operator T_{θ} : ℝ^{2} → ℝ^{2} given by

[T_{θ}] = [ \begin{matrix}cos (θ) & − sin(θ) \\ sin(θ) & cos (θ) \end{matrix} ]

is a (counter-clockwise) rotation around the origin through the angle θ. If θ is not an integral multiple of π, then T_{θ} has no eigenvalues.

• More generally, consider a non-zero linear operator T : ℝ^{2} → ℝ^{2}. If 0 ≠ u ∈ ℝ^{2}, then there must be a linear combination of {u, T (u), T^{2}(u)} with

0 = T^{2}(u) + aT (u) + b(u) = P_{u}(T)(u).

If P_{u}(T) factors into linear factors P_{u}(T) = (T − λ_{1}I)(T − λ_{2}I) with λ_{1}, λ_{2} ∈ ℝ, then λ_{1} and λ_{2} are eigenvalues of T. On the other hand, if a^{2} − 4b < 0, then T has no eigenvalues (it is a general rotation in ℝ^{2}).

• On the other hand, every non-zero linear operator T : ℂ^{n} → ℂ^{n} has an eigenvalue. We can see this from the fact that, for any u ≠ 0 in ℂ^{n}, the set of vectors {u, T (u), T^{2}(u), …, T^{n}(u)} must be a linearly dependent set, and hence there are a_{0}, a_{1}, …, a_{n} not all zero with

0 = n ; ∑ ; i=0 a_{i}T^{i}(u) = P_{u}(T)(u).

We know that we can factor the polynomial P_{u}(T) over ℂ into a product of linear factors

P_{u}(T) = _{j}^{∏}(T − z_{j}I)^{k_{j}}(T).

This linear operator is not a monomorphism (since u ∈ ker (P_{u}(T))). It is a product of commutative factors, and hence at least one of the factors (T − z_{j}I) is not a monomorphism. This says then that at least one of the z_{j} is an eigenvalue of T. We can proceed to find n eigenvalues and eigenvectors.

• For for every non-zero linear operator T : ℂ^{n} → ℂ^{n}, there is a polynomial P_{T} (T) = ∑_{i} a_{i}T^{i} of degree n, called the characteristic polynomial of T, with:

1. P_{T} (T) = 0. That is, P_{T} (T)(u) = 0 for all u ∈ ℂ^{n}.

2. The linear factors of P_{T} (T) are each of the form (T − λ_{j}I) for some eigenvalue λ_{j}. Also, a_{n} = 1. That is,

P_{T} (T) = _{j}^{∏}(T − λ_{j}I)^{k_{j}}(T).

3. For each eigenvalue λ_{j}, there is an eigenvector u_{j}. If k_{j} > 1, there is a linearly independent set of k_{j} eigenvectors for λ_{j}. The set of n eigenvectors {u_{i}} is a linearly independent set, and thus is a basis for ℂ^{n}. As above, the matrix representation of T with respect to this basis is [T] = diag (λ_{1}, …, λ_{n}) (with repetitions as necessary).

• A non-zero linear operator T : ℝ^{n} → ℝ^{n} also has a characteristic polynomial of degree n. The big difference for the real case is that the polynomial will factor into a combination of linear and quadratic factors. There is then a basis consisting partially of eigenvectors and partially of pairs of basis vectors for two-dimensional subspaces on which T is a rotation. The matrix of T with respect to this basis then looks like

[T] = diag (λ_{1}, …, λ_{k}, A_{k+1} , …, A_{(n−k)/2} ) where each A_{i} is a two-by-two matrix without eigenvalues.

• The set of eigenvalues of an operator is sometimes called the spectrum of the operator. In cases like ℂ^{n} where the eigenvectors form a basis for the space, using such as basis is sometimes known as the spectral decomposition of the operator.

• Just briefly, let’s glance at differential operators. D : C^{∞}(ℝ) → C^{∞}(ℝ) has uncountably many eigenvalues. Each real number a ∈ ℝ is an eigenvalue since

D(e^{ax}) = ae^{ax}.

The corresponding eigenvectors are of course f_{a}(x) = e^{ax}.

The differential operator D^{2} also has uncountably many eigenvalues and eigenvectors since for a > 0,

D^{2}(cos (\sqrt{a}x)) = −a cos (\sqrt{a}x),
D^{2}(sin(\sqrt{a}x)) = −a sin(\sqrt{a}x), and
D^{2}(e^{\sqrt{a}x}) = ae^{\sqrt{a}x}.

• In this context, here’s an example which puts many of these pieces together. In quantum mechanics, a typical expression of Schrödinger’s equation looks like

[−\frac{ℏ^{2}}{2m_{e}} ▽^{2} +V (x, y, z) − iℏ\frac{∂}{∂t}] Ψ = 0.

This example is for an electron (with mass m_{e}) in a potential field V (x, y, z).

The general solution of this operator equation is

Ψ(x, y, z, t) = ∞ ; ∑ ; n=0 c_{n}Ψ_{n}(x, y, z) exp (\frac{−iE_{n}t}{ℏ})

where Ψ_{n}(x, y, z) is an eigenfunction solution of the associated time independent Schrödinger equation, with E_{n} the corresponding eigenvalue. The inner product, giving a time dependent probability, looks like

P (t) = ∫ ΨΨ̄ dv.

Exercises: Eigenvectors and eigenvalues ←

1. Show that if v is an eigenvector of T corresponding with the eigenvalue λ, and α ∈ 𝔽 (α ≠ 0), then αv is also an eigenvector of T corresponding with λ. In particular, eigenvectors are not unique.

2. Define T ∈ L(𝔽^{2}) by T ((x, y)) = (y, x). Find all the eigenvalues and eigenvectors of T.

3. Define T ∈ L(𝔽^{3}) by T ((x, y, z)) = (−y, 0, 2z). Find all the eigenvalues and eigenvectors of T.

4. Define T ∈ L(𝔽^{∞}) by T ((a_{1}, a_{2}, …)) = (a_{2}, a_{3}, …) (i.e., T is the left shift operator). Find all the eigenvalues and eigenvectors of T.

5. Suppose T ∈ L(V) is invertible. Show that λ ≠ 0 is an eigenvalue of T if and only if λ^{−1} is an eigenvalue of T^{−1}.

6. Suppose S, T ∈ L(V). Show that ST and T S have the same eigenvalues.

7. Give an example of an operator whose matrix has all zeros on the diagonal with respect to some basis, but which is invertible. Give an example of an operator whose matrix has all diagonal elements non-zero with respect to some basis, but which is singular (i.e., has no inverse).

8. Show that if S, T ∈ L(V), S is invertible, and P (x) ∈ 𝔽[x], then P (S^{−1}T S) = S^{−1}P (T)S.

9. Suppose that V is a finite dimensional normed complex vector space, and T is an isometry of V (i.e., ||T (v)|| = ||v|| for all v ∈ V). Show that every eigenvalue λ of T has |λ| = 1. Hint: show that there is a basis for V consisting of eigenvectors e_{i}, with ||e_{i}|| = 1 for 1 ≤ i ≤ n.

10. Let V be a complex vector space, and T ∈ L(V). Let P (x) ∈ ℂ[x]. Show that α ∈ ℂ is an eigenvalue of P (T) if and only if α = P (λ) for some eigenvalue λ of T.

11. Is the preceding true for real vector spaces? Why not? (Note: this is another example of why ℂ is so much nicer a place to work than ℝ …)

Is the preceding true if we replace ℂ[x] with ℂ^{∞}[x], power series? If so, show it. If not, give a counterexample.

Change of basis ←

• Recall that the matrix associated with a linear transformation depends on the particular bases we use. In the case of a linear operator T : V → V, it is typical to use the same base for V as domain and as codomain. However, as we have seen, a basis consisting of eigenvectors is particularly convenient, and hence it is useful to know how to change bases.

If V is finite dimensional, with two bases S_{1} = (u_{1}, …, u_{n}) and S_{2} = (v_{1}, …, v_{n}), we can consider the matrices of T with respect to the mixed bases. We can indicate this by the various symbols [T]^{S_{1}}, [T]^{S_{2}}, [T]^{S_{1}S_{2}}, and [T]^{S_{2}S_{1}}, where, for example

T (u_{i}) = _{j}^{∑}[T]_{ji}^{S_{1}} u_{j}, and
T (u_{i}) = _{j}^{∑}[T]_{ji}^{S_{1}S_{2}} v_{j}.

• We can look in particular at the matrix representations for the identity operator I. For ease of notation, let [B] = [I]^{S_{1}S_{2}}, the matrix representation for I using S_{1} for the domain and S_{2} for the codomain. In particular, we then have

u_{i} = _{j}^{∑}[B]_{ji}v_{j} = B_{i}^{j}v_{j}.

We also have that the matrix inverse [B]^{−1} = [I]^{S_{2}S_{1}} which takes us in the opposite direction. Putting the pieces together, we then have

[B][T]^{S_{1}} = [T]^{S_{2}}[B].

That is, doing the operator T in the basis S_{1} and then converting to the basis S_{2} is the same as converting bases first, and then doing T in the second base.

Another way to say this is:

[T]^{S_{1}} = [B]^{−1}[T]^{S_{2}}[B]

or

[T]^{S_{2}} = [B][T]^{S_{1}}[B]^{−1}.

Exercises: Change of basis ←

1. Show that if A and B are operators on a finite dimensional vector space with AB = I, the identity operator, then BA = I, and so B = A^{−1}.

2. Suppose that T is an operator that has the same matrix with respect to every basis. Show that T must be some multiple of the identity operator I.

Trace and determinant ←

• Let V be a finite dimensional vector space over ℝ or ℂ, and T : V → V an operator on V. Let P_{T} (T) = ∑_{i=0}^{n} a_{i}T^{i} be the characteristic polynomial of T. We can then define the trace of T by trace (T) = a_{n−1}, the coefficient of T^{n−1}.

We can also define the trace of an nxn real or complex matrix A by trace _{m}(A) = ∑_{i=1}^{n} A_{ii}, the sum of the diagonal elements of A.

These definitions are consistent, in the sense that trace (T) = trace _{m}([T]), where [T] is the matrix of T with respect to some basis for V. An exercise will be to show that it doesn’t matter what basis we use (they all come out the same). Since these definitions are consistent, we will ordinarily write them both the same way, as trace ().

• For S, T : V → V, and a, b ∈ 𝔽, the trace has the nice properties:

1. trace (aS+bT) = a trace (T)+b trace (T). This says that the trace is a linear functional on L(V).

2. In the complex case, we have

trace (T) = _{i}^{∑} λ_{i}

where the λ_{i} are the eigenvalues of T.

3. trace (ST) = trace (T S).

• From these we can derive nice facts like:

If we define the commutator of S and T by [S, T] = ST − T S, then we always have [S, T] ≠ I, the identity operator.

• We can define the determinant of an operator T by det (T) = (−1)^{n}a_{0}, where a_{0} is the constant term in the characteristic polynomial.

• The determinant has the nice properties:

1. In the complex case, we have

det (T) = _{i}^{∏} λ_{i}

where the λ_{i} are the eigenvalues of T.

2. det (T) ≠ 0 iff ker (T) = {0} iff T is invertible.

3. det (ST) = det (S) det (T) = det (T S).

4. P_{T} (z) = det (zI − T). That is, the characteristic polynomial is the determinant of the generalized operator (zI − T), where we think of z as a complex variable.

5. We can define the determinant of a square matrix M = [a_{ij}] by

det_{m}(M) = _{p∈perm (n)}^{∑} sign (p) _{i}^{∏} a_{p(i)i}

where perm (n) is the set of all permutations of the numbers (1, 2, …, n), sign (p) is the sign of the permutation, and p(i) is the ith number in the permutation p. We then have:

det_{m}([T]) = det (T).

As for the trace, since the two definitions are consistent, we will typically denote both determinants by the same symbol det (). Showing that these two definitions are consistent is a fair amount of work (just looking at the formula for det_{m}() should give you some idea). You can look it up if you are interested.

6. There is another definition of determinant. Suppose V is a real or complex n-dimensional vector space. Then there is an alternating multi-linear functional

det_{v} : V^{n} → 𝔽

with the properties, for all v_{1}, v_{2}, …, v_{n}, v_{i_{1}}, v_{i_{2}} ∈ V and a, b ∈ 𝔽:

(a) det_{v}(v_{1}, …, v_{i}, …, v_{j}, …, v_{n}) = −det_{v}(v_{1}, …, v_{j}, …, v_{i}, …, v_{n}) (alternating)

(b) det_{v}(v_{1}, …, av_{i_{1}} + bv_{i_{2}}, …, v_{n}) = adet_{v}(v_{1}, …, v_{i_{1}}, …, v_{n}) + bdet_{v}(v_{1}, …, v_{i_{2}}, …, v_{n}) (multi-linear)

(c) det_{v}(e_{1}, …, e_{n}) = 1 for a particular basis {e_{1}, …, e_{n}} for V. (uniqueness)

We then have det_{m}(A) = det_{v}(A_{1}, …, A_{n}), where A_{k} is the kth column of the matrix A, considered as a vector. We have that det (T) = det_{m}([T]) = det_{v}([T]_{1}, …, [T]_{n}). The fact that there are three different versions of the same thing suggests that many people have worked on this topic, and that this topic occurs in a variety of contexts …

7. Recall the discussion of norms on vector spaces. The Euclidean norms (||v||_{2}), which can be thought of as coming from an inner product (< v_{1}, v_{2} >), have a nice relationship with the determinant:

||T (v)||_{2} = ||v||_{2} for all v ∈ U

iff | det (T)| = 1.

8. An operator which preserves the Euclidean norm (||T (v)||_{2} = ||v||_{2}) is called an isometry. When the scalar field is ℝ, these are called orthogonal operators. In the case of ℂ, they are called unitary operators. In the case of the quaternions, they are called symplectic operators.

9. The isometries on a normed linear space have the properties:

(a) If T is an isometry, then det (T) ≠ 0, and hence T has an inverse, T^{−1}, which is also an isometry. Of course, I, the identity operator, is an isometry.

(b) If T_{1} and T_{2} are isometries, then T_{1}T_{2} is also an isometry.

(c) This means that the set of isometries forms a group. In the three cases described above, these are called the orthogonal group, the unitary group, and the symplectic group.

10. Another way to characterize the isometries (in the finite dimensional case, but also with appropriate generalizations in infinite dimensional cases) is that an operator is an isometry if 1.) its inverse exists, and 2.) the matrix of its inverse is given by the conjugate transpose. That is, if

[T^{−1}]_{ij} = \overline{[T]_{ji}}.

11. The determinant also plays an important role in change of variables formulas (in ℝ^{n}, for example). Suppose Ω ⊂ ℝ^{n} and σ : Ω → ℝ^{n}. We will think of σ as a (local) coordinate system on Ω, or as a change of variables. The derivative of σ at x is the unique operator T (if it exists) satisfying:

lim_{y→0} \frac{||σ(x + y) − σ(x) − T y||}{||y||} = 0.

If this operator T exists, σ is called differentiable, and T is denoted by σ′(x). Note that σ′(x) ∈ L(ℝ^{n}). We can write σ(x) = (σ_{1}(x), …, σ_{n}(x)), and we can denote the partial derivative of σ_{j} with respect to the kth coordinate by D_{k}σ_{j}(x). If σ is differentiable at x, then the matrix of σ′(x) is given by:

[σ′(x)] = [ \begin{matrix}D_{1}σ_{1}(x) & … & D_{n}σ_{1}(x) \\ ⋮ & & ⋮ \\ D_{1}σ_{n}(x) & … & D_{n}σ_{n}(x) \end{matrix} ]

If we assume that Ω is a reasonable set (e.g., open, or measurable) and f : Ω → ℝ is also reasonable (e.g., continuous, or measurable), then we have the change of variables formula:

∫_{σ(Ω)} f(y)dy = ∫_{Ω} f(σ(x))| det (σ′(x))|dx.

Exercises: Trace and determinant ←

1. Given two real or complex nxn matrices A and B, show that trace _{m}(AB) = trace _{m}(BA).

2. Given an operator T on a real or complex finite dimensional vector space V, show that trace _{m}([T]) and det_{m}([T]) are independent of the basis used for the matrix representation [T]. Hint: use the change of basis formula [T]^{S_{1}} = [B]^{−1}[T]^{S_{2}}[B], the previous problem, and det_{m}(AB) = det_{m}(BA).

3. Show that trace is linear, that is, trace (aS + bT) = atrace (S) + btrace (T).

4. Show that trace (T) = ∑_{i=1}^{n} [T]_{ii}.

5. For two operators S and T, show that [S, T] ≠ I. Recall that [S, T] = ST − T S.

6. Show by example that in general, trace (ST) ≠ trace (S)trace (T). In particular, find an operator T on a real vector space V with trace (T^{2}) < 0.

7. Suppose A is an nxn real matrix, S ∈ L(ℝ^{n}) has matrix representation A with respect to some basis, and T ∈ L(ℂ^{n}) has matrix representation A with respect to some basis. Show that trace (S) = trace (T) and det (S) = det (T).

8. Show that if T is an isometry on a finite dimensional normed vector space, then | det (T)| = 1.

9. Show that if T is an operator on a complex vector space of dimension n, and α ∈ ℂ, then det (αT) = α^{n} det (T).

To top ←

References

[1] Axler, Sheldon, Linear Algebra Done Right, second edition, Springer-verlag, New York, 1997.

[2] Finney, Ross L., and Ostberg, Donald R., Elementary Differential Equations with Linear Algebra, Addison-Wesley, 1976.

[3] Hoffman, Kenneth, and Kunze, Ray, Linear Algebra, second edition, Engineering/Science/Mathematics, 1971.

[4] Naimark, M. A., Normed Algebras, Wolters-Noordhoff, Groningen, the Netherlands, 1974.

[5] Noble, Ben, and Daniel, James W., Applied Linear Algebra, third edition, Engineering/Science/Mathematics, 1988.

[6] Prugovečki, Eduard, Quantum Mechanics in Hilbert Space, second edition, Academic Press, New York, 1981.

[7] Rudin, Walter, Functional Analysis, McGraw-Hill, New York, 1973.

[8] Strang, Gilbert, Linear Algebra and its Applications, Academic Press, New York, 1976.

To top ←