A brief survey of linear algebra

Tom Carter

http://cogs.csustan.edu/~tom/linear-algebra

Santa Fe Institute
Complex Systems Summer School

June, 2001

Our general topics: Top

• Why linear algebra

• Vector spaces    (ex)

• Examples of vector spaces    (ex)

• Subspaces    (ex)

• Linear dependence and independence    (ex)

• Span of a set of vectors    (ex)

• Basis for a vector space    (ex)

• Linear transformations    (ex)

• Morphisms - mono, epi, and iso    (ex)

• Linear operators    (ex)

• Normed linear spaces    (ex)

• Eigenvectors and eigenvalues    (ex)

• Change of basis    (ex)

• Trace and determinant    (ex)

• References

(ex): exercises.

Why linear algebra Top

• Linear models of phenomena are pervasive throughout science. The techniques of linear algebra provide tools which are applicable in a wide variety of contexts.

  Beyond that, linear algebra courses are often the transition from lower division mathematics courses such as calculus, probability/statistics, and elementary differential equations, which typically focus on specific problem solving techniques, to the more theoretical axiomatic and proof oriented upper division mathematics courses.

  I am going to stay with a generally abstract, axiomatic presentation of the basics of linear algebra. (But I'll also try to provide some practical advice along the way ... :-)

• The first thing we need is a field F of coefficients for our vector space. The most frequently used fields are the real numbers R and the complex numbers C. A field F = (F, +, *, -,  ^-1, 0, 1) is a mathematical object consisting of a set of elements (F), together with two binary operations (+, *), two unary operations (-,  ^-1), and two distinguished elements 0 and 1 of F, which satisfy the fundamental properties:

  1. F is closed under the four operations:

     + : Fx F® F * : Fx F® F - : F® F -1 : F¢® F¢    F¢ = {a Î F| a ¹ 0}

     Of course, we usually write a + b, a * b, -a, and a^-1 instead of +(a,b), *(a, b), -(a), and  ^-1(a).

  2. `+' and `*' are commutative and associative, and satisfy the distributive property. That is, for a, b, c Î F:

     a + b = b + a a * b = b * a (a + b) + c = a + (b + c) (a * b) * c = a * (b * c) a * (b + c) = a * b + a * c

  3. 0 is the identity element and `-' is the inverse for addition. 1 is the identity element and `a^-1' is the inverse for multiplication. That is, for a Î F:

     a + 0 = a a + (-a) = 0 a * 1 = a a * (a-1) = 1     (a ¹ 0)

  4. Although we won't need it for most of linear algebra, I'll mention that R and C are both complete (Cauchy sequences have limits), and R is fully ordered (a < b or b < a or a = b for all a, b Î R).

  5. As needed, we will identify R as a subfield of C, and we will typically write elements of C as a + bi, where a and b are real and i² = -1.

  6. Although R and C are the most frequently used coefficient fields, there are many other fields such as Q, the rationals, and the finite fields Z/pZ for p a prime.

• If we leave out the requirement that a * b = b * a, we get what are called skew fields. An important example of a skew field is H, the quaternions (also called the hamiltonians) which contains additional square roots of -1, i² = j² = k² = -1, and ij = k, jk = i, ki = j, but ij = -ji, jk = -kj, and ki = -ik. We typically write quaternions either in the form a + bi + cj + dk with a, b, c, d Î R, or
  a+ bj with a, b Î C.

• We are then ready for the definition. A vector space V = (V, F, +, -, *, 0) over a field F is a set V of elements (called vectors) together with a distinguished element 0, two binary operations, and a unary operation:

  + : V x V ® V - : V ® V * : Fx V ® V

  For u, v, w Î V, and a, b Î F, these operations satisfy the properties:

  v + w = w + v (u + v) + w = u + (v + w) v + 0 = v v + (-v) = 0 1 * v = v a * (u + v) = (a * u) + (a * v) (a + b) * v = (a * v) + (b * v) (a * b) * v = a * (b * v)

  The elements of the field F are called the scalars, or coefficients of the vector space.

• From the basic properties listed above, we can prove a variety of additional properties, such as:

  0 * v = 0 a * 0 = 0 (-1) * v = -v

  We can also prove that the additive identity 0 is unique, as is the additive inverse -v.

• We will usually simplify the notation, and write av instead of a * v. Furthermore, although it is important to distinguish the scalar 0 from the vector 0, we will typically write the vector in the simple form 0.

• If F = R, we call V a real vector space, and typically write the scalars as a, b, c. If F = C, we call V a complex vector space, and often write the scalars as a, b, g.

Exercises: Vector spaces Top

1. Using the basic properties listed above, prove the additional properties:

   1. 0 * v = 0
   2. a * 0 = 0
   3. (-1) * v = -v
   4. -(-v) = v
   5. For a Î F and v Î V, av = 0 iff a = 0 or v = 0.

2. Prove that the additive identity 0 is unique.
3. Prove that the additive inverse -v is unique.

Examples of vector spaces Top

• The first main example of a real vector space is Rⁿ, the Cartesian product of n copies of the real line. An element of Rⁿ looks like (a₁, a₂, ¼, a_n). When we add two vectors, we get

  (a1, a2, ¼, an) + (b1, b2, ¼, bn)

  = (a1+b1, a2+b2, ¼, an + bn).

  When we multiply by a scalar a Î R, we get

  a * (a1, a2, ¼, an) = (aa1, aa2, ¼, aan).

• Similary, we have the complex vector space Cⁿ, with vector addition and scalar multiplication defined the same way.

• Note that we can also think of Cⁿ as a real vector space, if we restrict the scalars to R.

• Let F^¥ = {(a₀, a₁, ¼)}, with
  (a₀, a₁, ¼) + (b₀, b₁, ¼)
          = (a₀ + b₀, a₁ + b₁, ¼),
  and a * (a₀, a₁, ¼) = (a*a₀, a*a₁, ¼).
  This is a vector space over F.

• For a scalar field F, we can define F[x] to be the set of all polynomials with formal variable x, and coefficients in F. This is a vector space over F, the coefficient field. Each polynomial is a vector. Vector addition is just polynomial adddition, and scalar multiplication just multiplies each coefficient in the polynomial by the scalar:

  a * (a0 + a1x1 + ¼anxn)

  = aa0 + aa1x1 + ¼aanxn.

• For a scalar field F, let Fⁿ[x] be the set of all polynomials of degree £ n with coefficients in F. We can also let F^¥[x] be the vector space of (formal) power series over F. These are also vector spaces over F.

• Let C⁰(R) be the set of all continuous functions with domain and range the real numbers. That is:
  C⁰(R) = {f:R® R | f is continuous}.
  We can define an addition operation on this set. To specify the sum of two functions, we must specify the sum at each point in the domain. If f, g Î C⁰(R), we define f + g for each x by

  (f + g)(x) = f(x) + g(x).

  We define scalar multiplication pointwise also: (a*f)(x) = a*(f(x)).

  C⁰(R) thus becomes a real vector space, where each continuous function is a vector in the space.

• Let Cⁿ(R) be the set of all continuous real functions whose first n derivatives are also continuous, and define addition and scalar multiplication pointwise. Similarly, we can define C^¥(R) as functions with all derivatives continuous. Again we get real vector spaces.

• Let C⁰[0,1] be the set of all continuous functions with domain the closed interval [0,1], and range R. We can also define Cⁿ[0,1] to be those functions with first n derivatives continuous, and C^¥[0,1] with all derivatives continuous. We can also generalize to subsets of R other than the interval [0,1]. With pointwise addition and scalar multiplication, these are each real vector spaces.

• We get can get similar complex vector spaces if we use C instead of R in examples like those above.

Exercises: Examples of vector spaces Top

1. Show that each of the examples listed in this section is a vector space.

2. Consider the set of points in R² of the forms (x, 0), x Î R, and (0, y), y Î R (i.e., the union of the X-axis and the Y-axis). Show that with the usual vector addition in R², this is not a vector space over R.

3. Consider the set R² with usual vector addition:

   (x1, y1) + (x2, y2) = (x1 + x2, y1 + y2),

   but with ``scalar multiplication'' given by

   a * (x, y) = (ax, y)

   for a Î R. Show that this is not a vector space over R.

Subspaces Top

• If V is a vector space over F, a subset U Ì V is called a vector subspace of V if U is a vector space over F using the same vector addition and scalar multiplication as in V. Often, in context, we will just call U a subspace of V.

• Most often, given a subset U Ì V, we will just let U inherit the vector addition and scalar multiplication operations from V. Hence, most of the vector space properties will come for free. What may not come for free, however, is whether the subset U is closed under the operations.

  Thus, when we inherit the operations from V, we will have that

  + : U x U ® V,

  and

  * : Fx U ® V,

  whereas, what we need is

  + : U x U ® U,

  and

  * : Fx U ® U.

  In order for U to be a vector subspace, we must be sure that doing the operations will leave us in the subset U.

• Examples of subspaces:

  1. {0} is a subspace of V for any vector space V.
  2. If m £ n, then F^m (identified with {(a₁, a₂, ¼, a_m, 0, ¼, 0)}) is a subspace of Fⁿ.
  3. If m £ n, then F^m[x] is a subspace of Fⁿ[x], and a subspace of F[x].
  4. If m £ n, then Cⁿ(R) and C^¥(R) are subspaces of C^m(R) .
  5. In R², let U = {(x, y) | y = 3x}. Then U is a subspace.
  6. More generally, in Rⁿ, fix a₁, a₂, ¼, a_n, and let U = {(x₁, x₂, ¼, x_n) |      a₁x₁ + a₂x₂ + ¼+ a_nx_n = 0}.
     Then U is a subspace.
  7. If U₁ and U₂ are subspaces, so is U₁ ÇU₂.

• Examples which are not subspaces:

  1. In R², let U = {(x, y) | x = 0 or y = 0} (i.e., U is the union of the x-axis and the y-axis). Then U is not a subspace, since the two vectors (1,0) and (0,1) are in U, but (1,0) + (0,1) = (1,1) is not in U.
  2. In R², let U = {(x, y) | x and y are both rational numbers}. Then U is not a subspace, since (1, 1) is an element of U, but Ö2*(1, 1) = (Ö2, Ö2) is not an element of U.
  3. In Rⁿ, fix a₁, a₂, ¼, a_n, and let U = {(x₁, x₂, ¼, x_n) | a₁x₁ + a₂x₂ + ¼+ a_nx_n = 2}. Then U is not a subspace, since, for example, 0 is not an element of U.
  4. In general, if U₁ and U₂ are subspaces, U₁ ÈU₂ is not a subspace (except in special cases, such as when U₁ Ì U₂).

• If U₁ and U₂ are both subspaces of a vector space V, we can define the subspace which is the sum of the two subspaces by:

  U1 + U2 = {u1 + u2 | u1 Î U1 and u2 Î U2}.

• If in addition U₁ ÇU₂ = {0}, then each element u Î U₁ + U₂ can be written in a unique way as u = u₁ + u₂. In this case, we call the sum of U₁ and U₂ a direct sum, and write it U₁ ÅU₂.

• These ideas generalize in a straightforward way to U₁+¼+ U_n and U₁żÅU_n for a finite number of subspaces, and U₁+U₂+¼  and U₁ÅU₂ż for countably many subspaces.

Exercises: Subspaces Top

1. Show that each of the examples identified in this section as a subspace actually is a vector subspace.

2. Which of the following are vector subspaces of R³?

   1. {(x₁, x₂, x₃) Î R³ | 3x₁ + 2x₂ - x₃ = 0}
   2. {(x₁, x₂, x₃) Î R³ | 3x₁ + 2x₂ - x₃ = 4}
   3. {(x₁, x₂, x₃) Î R³ | x₁x₂x₃ = 0}

3. Suppose that U is a vector subspace of V, and V is a vector subspace of W. Show that U is a vector subspace of W.

4. Show that the intersection of any collection of vector subspaces of V is a vector subspace of V.

5. Let

   l2 = {(ai) Î R¥ | ¥ å i = 0  |ai|2 < ¥}.

   Show that l² is a vector subspace of R^¥.

6. Let

   L2 = {f Î C0(R) | ó õ ¥ -¥  |f(x)|2 dx < ¥}.

   Show that L² is a vector subspace of C⁰(R).

From here on, we'll assume that U and V are vector spaces over a field F.

• Suppose that v₁, v₂, ¼, v_n are vectors in V, and that for some a₂, a₃, ¼, a_n Î F,

  v1 = a2v2 + a3v3 + ¼+ anvn.

  Then we say that v₁ is linearly dependent on v₂, ¼, v_n. Note that if we move v₁ to the other side, and let a₁ = -1, we have a₁v₁ + a₂v₂ + ¼+ a_nv_n = 0, and not all of the a_i are zero (in particular, a₁ ¹ 0).

  This motivates a general definition: a set S of vectors in a vector space V is called linearly dependent if, for some n > 0, and distinct v₁, v₂, ¼v_n Î S, there exist a₁, a₂, ¼, a_n Î F, not all 0, with

  a1v1 + a2v2 + ¼+ anvn = 0.

• The flip side of this definition is the following: a set S of vectors in a vector space V is called linearly independent if, given distinct v₁, v₂, ¼v_n Î S, the only way for a₁, a₂, ¼, a_n Î F to give

  a1v1 + a2v2 + ¼+ anvn = 0

  is if every one of the a_i are zero.

  A useful way to think about this is that a set S of vectors is linearly independent if no individual one of the vectors is linearly dependent on a finite number of the rest of them.

• Some examples:

  1. In R², the sets of vectors {(1,0),(0,1)}, {(1,2),(1,3)}, {(1,0)} and {(2,2),(3,2)} are each linearly independent sets.
  2. In R², none of the sets of vectors {(1,0),(2,0)}, {(1,0),(0,1), (1,1)}, {(0,0),(0,1)}, {(0,0)}, {(1,2),(2,3),(3,4),(4,1)}, nor {(1,2),(3,4),(2,1)} is a linearly independent set.
  3. In any vector space, if a set of vectors contains the 0 vector, the set is not linearly independent.
  4. In C⁰(R), if f₁(1) = f₂(2) = ¼ = f_n(n) = 1, but f_i(j) = 0 for i ¹ j, then the set of functions {f₁, f₂, ¼, f_n} is linearly independent.
  5. In C^¥(R), the set of functions {cos(nx) | n = 0, 1, 2, ¼} is a linearly independent set.
  6. The empty set, {}, is a linearly independent set.

• Mathematics is an extremely precise language. These two sentences do not mean the same thing:

  1. The set {v₁, v₂, ¼, v_n} of vectors is linearly independent.
  2. {v₁, v₂, ¼, v_n} is a set of linearly independent vectors.

  One of the hardest parts of doing mathematics is developing your mathematical intuition. It is tempting to imagine that intuition is what you have before you know anything, but that is nonsense. Intuition is just the automatic part of your knowledge, derived from your past experience. Becoming better at mathematics involves learning new mathematics, and then integrating that new knowledge into your intuition. Doing that takes care, precision, and lots of practice!

Exercises: Linear dependence and independence Top

1. Verify the statements in each of the six examples in this section.

2. Suppose that P₀, P₁, ¼, P_n are polynomials in Fⁿ[x], and P_i(1) = 0 for i = 0, 1, ¼,n. Show that {P₀ , P₁, ¼, P_n} is not linearly independent in Fⁿ[x].

3. In C^¥(R), show that each of these sets of functions is linearly independent:
   1. {xⁿ | n = 0, 1, 2, ¼}.

   2. {sin(nx) | n = 0, 1, 2, ¼}.

   3. {e^nx | n = 0, 1, 2, ¼}.

• If S is a set of vectors in the vector space V over F, we define the span of the set S by:

  span(S) = {a1v1 + a2v2 + ¼+ anvn |

  ai Î F, vi Î S, n > 0}.

  This says that span(S) is the set of all vectors that can be written as finite linear combinations of the vectors in S.

• Examples:

  1. In R¹, span({(1)}) is all of R¹.

  2. In R³, span({(1,0,0), (0,1,0)}) is the x-y plane. Similarly, span({(1,0,0), (0,0,1)}) is the x-z plane, span({(0,1,0), (0,0,1)}) is the y-z plane, and span({(1,0,0), (0,1,0), (0,0,1)}) is all of R³.

  3. Still in R³, span({(1,2,0), (2,1,0)}) is the x-y plane, span({(1,0,0)}) is the x axis, span({(1,0,0), (1,1,0), (1,1,1)}) is all of R³, span({(1,2,3), (1,1,0), (1,1,1), (2,1,4)}) is all of R³, and span({(1,2,0), (1,1,0), (3,1,0), (2,1,0)}) is the x-y plane.

  4. In Fⁿ[x], span({1, x¹, x², ¼, xⁿ}) = Fⁿ[x].

  5. In F[x], span({1, x¹, x², ¼}) = F[x].

  6. In C⁰(R), span({f₁(x) = 1, f₂(x) = x}) is the set of all linear functions, y = mx + b.

  7. For any set S, span(S) is a subspace.

  8. In particular, span(S) is sometimes defined to be the smallest subspace of V containing S, or the intersection of all subspaces of V that contain S.

1. Verify the statements in each of the eight examples in this section.

2. Show that if span({v₀, v₁, ¼, v_n}) = V, then span({v₀ - v₁, v₁ - v₂, ¼,v_n-1 - v_n, v_n}) = V.

• Suppose S is an ordered set (or list) of vectors in a vector space V. Suppose further that:

  1. S is linearly independent, and
  2. span(S) = V.

  Then we call S a basis for the vector space V.

• Given a basis S for the vector space V, every vector v Î V, v ¹ 0, can be written in a unique way as:

  v = a1v1 + a2v2 + ¼+ anvn

  with a₁, a₂, ¼, a_n Î F, a_i ¹ 0, and v₁, v₂, ¼, v_n Î S. For uniqueness, we take the v_i to be distinct, and to be in the order they appear in S. We are writing each v as a finite linear combination of the basis vectors.

• Examples:

  1. In F³, the ordered set S = ((1,0,0),(0,1,0),(0,0,1)) is a basis (often called the standard basis). This generalizes in the obvious way to Fⁿ.

  2. In F³, the ordered set S = ((1,0,0),(1,1,0),(1,1,1)) is a basis. So is ((1,2,3),(2,1,3),(1,2,2)).

  3. In general, in Fⁿ any linearly independent ordered set S = (v₁, v₂, ¼, v_n) of size n is a basis.

  4. (1, x, x², x³, ¼) is a basis for F[x].

  5. If S is a linearly independent ordered set, then S is a basis for span(S).

• If a vector space V has a finite basis S = (v₁, v₂, ¼, v_n), we say that V is finite dimensional, and we define the dimension of V by:

  dim(V) = n.

  We define dim({0}) = 0.

  Note that if a vector space V has a finite basis of size n, then every basis for V contains n vectors, and thus the definition makes sense.

  For example, dim(Fⁿ) = n for any field F and n > 0.

  We also have that dim(Fⁿ[x]) = n + 1.

• If a vector space V has a finite or countably infinite basis S, then we can uniquely represent each vector v with respect to S by a list of the form:
  v = (b₁, b₂, ¼, b_n) in the finite case, or
  v = (b₁, b₂, ¼) in the infinite case.

  Each b_i is either the non-zero coefficient corresponding with the ith element of S from the unique representation described above, or 0 if the basis element does not appear there. In the infinite case, only finitely many of the b_i are non-zero.

  We represent 0 by (0, 0, ¼, 0) in the finite case, and by (0, 0, ¼) in the infinite case.

• In these cases, we also have, for v_i Î S, that

  V = span(v1)żÅspan(vn)

  in the finite case, and

  V = span(v1)Åspan(v2)ż

  in the countably infinite case.

1. Verify the statements in each of the five examples in this section.

2. Let U be the subspace of R⁶ given by U = {(x₁, x₂, x₃, x₄, x₅, x₆) Î R⁶ |  2x₁ + 3x₃ = 0 and x₂ = 4x₅}.

   Find a basis for U.

3. Show that if U₁ and U₂ are subspaces of a finite dimensional vector space, then dim(U₁ + U₂) = dim(U₁) + dim(U₂) - dim(U₁ ÇU₂).

• If U and V are vector spaces over F, then a function T : U ® V is called a linear transformation or linear mapping if

  T(a1u1 + a2u2) = a1T(u1) + a2T(u2)

  for all a₁, a₂ Î F and u₁, u₂ Î U.

  An equivalent pair of conditions is that T(u₁ + u₂) = T(u₁) + T(u₂), and T(au) = aT(u).

• For a linear transformation T : U ® V, we call U the domain and V the codomain (or sometimes range) of T. We also define the kernel (or null space) of T by:

  ker(T) = {u Î U | T(u) = 0}.

  Further, we define the image (or sometimes range - be careful here!) of T by:

  im(T) = {v Î V | v = T(u) for some u Î U}.

• For a linear transformation T : U ® V we have the nice properties:

  1. ker(T) is a subspace of U, and im(T) is a subspace of V. (ex)
  2. If U is finite dimensional, then

     dim(U) = dim(ker(T)) + dim(im(T)).

     (ex)
  3. If U is finite dimensional with basis (u₁, u₂, ¼, u_n), and V is finite dimensional with basis (v₁, v₂, ¼, v_m), then T is determined by its effect on the basis elements u_i. There exist a_ij, 1 £ i £ m, 1 £ j £ n with:

     T(uj) = a1jv1 + a2jv2 + ¼+ amjvm,

     and in general, if u = å_jb_ju_j, then

     T(u) = m å i = 1  n å j = 1  aijbjvi.

  4. Thus, given particular bases for U and V, we can represent the linear transformation T by the matrix

     [T] = é ê ê ê ê ë a11 a12 ¼ a1n a21 a22 ¼ a2n : : : : am1 am2 ¼ amn ù ú ú ú ú û

     or [T]_ij = a_ij.
  5. If we represent the vector u Î U by the column matrix [b₁, b₂, ¼, b_n]^t, then we have

     [T(u)] = [T][u] = é ê ê ê ê ë a11 a12 ¼ a1n a21 a22 ¼ a2n : : : : am1 am2 ¼ amn ù ú ú ú ú û é ê ê ê ê ë b1 b2 : bn ù ú ú ú ú û = é ê ê ê ê ë a11b1 + a12b2 + ¼+ a1nbn a21b1 + a22b2 + ¼+ a2nbn : am1b1 + am2b2 + ¼+ amnbn ù ú ú ú ú û

  6. Sometimes it is more convenient to use the Einstein summation conventions, where we would write:

     T(u)i = Tijuj

     with implied summation over the repeated upper and lower indices.

  7. If T₁ : U ® V and T₂ : U ® V are two linear transformations, we can define the sum of the two transformations as

     (T1 + T2)(u) = T1(u) + T2(u),

     and we can define scalar multiplication of a linear transformation T by a as

     (a * T)(u) = a * (T(u)).

     We can thus define L(U, V) to be the space of all linear transformations from U to V. We define the zero transformation 0 : U ® V by 0(u) = 0. With these definitions, L(U, V) is a vector space. (ex)

  8. Given particular bases for U and V, the matrix representation of T₁ + T₂ is given by: [T₁ + T₂] = [T₁] + [T₂].

  9. If U and V are finite dimensional, then L(U, V) is also finite dimensional, and

     dim(L(U, V)) = dim(U)*dim(V).

     (ex)

  10. If T : U ® V and S : V ® W are linear transformations, then the composition S °T : U ® W is also a linear transformation. (ex)

      Recall that (S °T)(u) = S(T(u)).

      Given particular bases for U, V, and W, the matrix representation of S °T is the matrix product [S °T] = [S][T]. We usually abbreviate S °T as ST.

      It is worth noting that unless W Í U, it doesn't even make sense to talk about T °S.

  11. The matrix of the composition of two linear transformations is given by the product of the two matrices, given by:

      [ST] = [S][T] =

      é ê ê ê ê ë a11 a12 ¼ a1n a21 a22 ¼ a2n : : : : am1 am2 ¼ amn ù ú ú ú ú û é ê ê ê ê ë b11 b12 ¼ b1p b21 b22 ¼ b2p : : : : bn1 bn2 ¼ bnp ù ú ú ú ú û

      =

      é ê ê ê ê ê ê ê ê ê ê ë n å i = 1  a1ibi1 n å i = 1  a1ibi2 ¼ n å i = 1  a1ibip n å i = 1  a2ibi1 n å i = 1  a2ibi2 ¼ n å i = 1  a2ibip : : : : n å i = 1  amibi1 n å i = 1  amibi2 ¼ n å i = 1  amibip ù ú ú ú ú ú ú ú ú ú ú û

  12. We also have the distributive property:

      S(T1 + T2) = (ST1) + (ST2).

      The matrix representation for this is:

      [S][T1 + T2] = [S][T1] + [S][T2].

• Some examples of linear transformations:
  1. The function T : R² ® R given by T((x, y)) = x + y is linear.

  2. The function T : R² ® R³ given by T((x, y)) = (x + 2y, x - 2y, x - y) is linear.

  3. Given a₁, a₂, ¼, a_n Î F, and V a finite dimensional vector space over F with basis (v₁, v₂, ¼, v_n), the function T : V ® F given by T(b₁v₁ + ¼+ b_nv_n) = a₁b₁ + ¼+ a_nb_n is linear.

     In general, for a vector space V, a linear transformation T : V ® F is called a linear functional. The study of these transformations is called functional analysis.

  4. The function D : Fⁿ[x] ® F^n-1[x] given by
     D(a_nxⁿ + ¼+ a₁x + a₀)
           = na_nx^n-1 + ¼+ 2a₂x + a₁
     is linear. This linear transformation is called the derivative ...

  5. Similarly, we have the derivative transformation D: F^¥[x] ® F^¥[x] given by

     D( ¥ å n = 0  anxn) = ¥ å n = 1  n*anxn-1.

  6. The shift map S : F^¥ ® F^¥ is given by: S((a₀, a₁, ¼)) = (a₁, a₂, ¼).

  7. The difference map D: F^¥ ® F^¥ is given by:

     D((a0, a1, ¼)) = (a1 - a0, a2 - a1, ¼),

     or, abbreviating (a₀, a₁, ¼) by (a_n),

     D((an)) = (an+1 - an).

1. Verify each of the statements marked (ex) in this section.

2. Verify that each of the 7 examples actually are linear transformations.

3. Show that the usual calculus derivative [d / dx] : C^¥(R) ® C^¥(R) given by:

   d f(x) dx = lim h® 0  f(x+h) - f(x) h

   is a linear transformation.

4. Is the usual calculus indefinite integral

   ó õ f(x)dx

   a linear transformation? Why or why not? What about the definite integral?

• In algebra, we call a function that preserves structure a morphism. In our current context, a linear transformation preserves the linear structure of a vector space in the sense that T(u₁ + u₂) = T(u₁) + T(u₂) and T(au) = aT(u). Thus, linear transformations are morphisms in the category of vector spaces.

• We will be particularly interested in morphisms that have additional properties. Specifically, we are likely to look for morphisms T : U ® V which have one or more of the properties:
  1. One-to-one. Such transformations are also called injective, or monomorphisms. These transformations have the property that if u₁ ¹ u₂, then T(u₁) ¹ T(u₂). This says that different things get sent different places - that is, no two things get sent to the same place.

     Monomorphisms are nice because the subspace im(T) Ì V looks just like U.

  2. Onto. Such transformations are also called surjective, or epimorphisms. These transformations have the property that for every v Î V, there exists some u Î U with T(u) = v. This says that every element of V is hit by some element of U. Another way to say this is that im(T) = V.

     Epimorphisms are nice, because the algebraic properties of V will be reflected back in U.

  3. Both one-to-one and onto. Such transformations are also called bijective. A bijective function f : X ® Y has an inverse f^-1 : Y ® X with the properties that for all x Î X and all y Î Y, f^-1(f(x)) = x and f(f^-1(y)) = y.

     A bijective morphism whose inverse also preserves algebraic structure is called an isomorphism. In linear algebra, we have the nice property that if a linear transformation is bijective, then its inverse is also linear, and thus it is an isomorphism.

  4. If there is an isomorphism T : U ® V, we say that the spaces U and V are isomorphic. If two spaces are isomorphic, then they share all relevant properties. Thus, two isomorphic vector spaces are indistinguishable as vector spaces except for a renaming of the elements.

• We have the nice fact that a linear transformation T : U ® V is a monomorphism (is one-to-one) if and only if ker(T) = {0}.

  Pf.: Suppose T is a monomorphism. We know that for every linear transformation, T(0) = 0. Then, since T is a monomorphism, we know that if T(u) = 0 = T(0), it must be that u = 0. Thus ker(T) = {0}.

  On the other hand, suppose that ker(T) = {0}. Then, if T(u₁) = T(u₂), we will have 0 = T(u₁) - T(u₂) = T(u₁ - u₂). But this means that u₁ - u₂ Î ker(T) and hence, since we are assuming that ker(T) = {0}, we must have u₁ - u₂ = 0, or u₁ = u₂. By the contrapositive, this means that if u₁ ¹ u₂, then T(u₁) ¹ T(u₂). Q.E.D.

  (I had to do at least one proof,
  didn't I? :-)

• An important example of an isomorphism is the identity transformation I_U : U ® U given by I_U(u) = u. (ex)

• If T : U ® V is a monomorphism, then there is a linear transformation S₁ : im(T) ® U with S₁(T(u)) = (S₁T)(u) = u for all u Î U. This says that S₁T = I_U. S₁ is called a left (partial) inverse of T. (ex)

• If T : U ® V is an epimorphism, then there is a linear transformation S₂ : V ® U with T(S₂(v)) = (TS₂)(v) = v for all v Î V. This says that TS₂ = I_V. S₂ is called a right (partial) inverse of T. (ex)

• If T : U ® V is an isomorphism, then since T is an epimorphism, both S₁ and S₂ (as above) exist. Also, im(T) = V. We therefore have that for all v Î V,

  S1(v) = S1((TS2)(v)) = (S1T)(S2(v)) = S2(v).

  Thus S₁ = S₂. In this case, the (two-sided) inverse of T exists, and we have T^-1 = S₁ = S₂.

• If U and V are finite dimensional vector spaces over F with dim(U) = dim(V), then U and V are isomorphic. (ex)

  (Big) hint for proof: Let (u₁, u₂, ¼, u_n) and (v₁, v₂, ¼, v_n) be bases for U and V respectively. Define T : U ® V by T(u_i) = v_i for 1 £ i £ n, and extend by linearity. Make sense of the phrase ``extend by linearity,'' and then show that T is an isomorphism.

• We also have the nice fact that if dim(U) = dim(V) = n, U, V over F, and T : U ® V is linear, then the following are equivalent: (ex)

  1. T is a monomorphism.
  2. T is an epimorphism.
  3. T is an isomorphism.

  This means, for example, that such a T is onto if and only ker(T) = 0.

• If T : U ® V and S : V ® W, then (ex)

  1. If both S and T are monomorphisms, then so is ST.
  2. If ST is a monomorphism, then so is T.
  3. If both S and T are epimorphisms, then so is ST.
  4. If ST is an epimorphism, then so is S.

• If T : V ® V and S : V ® V, and ST = TS, then (ex)

  1. Both S and T are monomorphisms if and only if ST is a monomorphism.
  2. Both S and T are epimorphisms if and only if ST is an epimorphism.

• Let matrix(F, n, m) be the space of all n by m matrices with entries from F. We use ordinary entry by entry matrix addition, where (A + B)_ij = A_ij + B_ij, scalar multiplication, where (aA)_ij = aA_ij, and let (0)_ij = 0. Then matrix(F, n, m) is an n*m-dimensional vector space over F. If dim(U) = m and dim(V) = n, then we can define

  mat : L(U, V) ® matrix(F, n, m)

  by mat(T) = [T].

  This transformation is an isomorphism. (ex)

Exercises: Morphisms - mono, epi, and iso Top

1. Verify each of the statements marked (ex) in this section.

2. Show that if a function has an inverse, then the inverse is unique.

3. Consider the function T : Fⁿ[x] ® Fⁿ⁺¹ given by

   T(a0 + a1x + ¼+ anxn) = (a0, ¼, an).

   Show that this function is an isomorphism.

4. Consider the function T : C® matrix(R, 2, 2) given by

   T(a + bi) = é ê ë a -b b a ù ú û

   1. Show that if we consider C as a real vector space, this function is a monomorphism.
   2. Show that this function also respects complex multiplication, that is,

      T((a + bi)(c + di)) = T(a + bi) T(c + di).

5. More generally, consider the function T : matrix(C, n, n) ® matrix(R, 2n, 2n) given by

   T æ ç ç ç è é ê ê ê ë a11 + b11i ¼ a1n + b1ni : : an1 + bn1i ¼ ann + bnni ù ú ú ú û ö ÷ ÷ ÷ ø

   =   é ê ê ê ê ê ê ê ê ë a11 -b11 ¼ a1n -b1n b11 a11 ¼ b1n a1n : : : : an1 -bn1 ¼ ann -bnn bn1 an1 ¼ bnn ann ù ú ú ú ú ú ú ú ú û

   1. Show that if we consider matrix(C, n, n) as a real vector space, this function is a monomorphism.
   2. Show that this function also respects matrix multiplication, that is,

      T(A B) = T(A) T(B).

6. What the heck. Let H be the quaternions (described above). Consider the function T : matrix(H, n, n) ® matrix(C, 2n, 2n) given by

   T æ ç ç ç è é ê ê ê ë a11 + b11j ¼ a1n + b1nj : : an1 + bn1j ¼ ann + bnnj ù ú ú ú û ö ÷ ÷ ÷ ø

   =   é ê ê ê ê ê ê ê ê ê ê ê ê ë a11 - b11   ¼ a1n - b1n   b11 a11 ¼ b1n a1n : : : : an1 - bn1   ¼ ann - bnn   bn1 an1 ¼ bnn ann ù ú ú ú ú ú ú ú ú ú ú ú ú û

   1. Show that if we consider matrix(H, n, n) as a complex vector space, this function is a monomorphism.
   2. Show that this function also respects matrix multiplication, that is,

      T(A B) = T(A) T(B).

   Thus, if we denote by O(n), U(n), and Sp(n) the distance preserving linear operators on Rⁿ, Cⁿ, and Hⁿ respectively (called the orthogonal, unitary, and symplectic groups), then we have the monomorphisms:

   ¼® O(n) ® Sp(n) ® U(2n) ® O(4n)

   ® Sp(4n) ® U(8n) ® O(16n) ® ¼

• If T : V ® V is a linear transformation, we call T a linear operator on V. Note that if S and T are operators on V, then so is ST. We can abbreviate L(V, V) as L(V).

  L(V) has the algebraic structure of a ring with identity. A ring is similar to a field (as defined above), except without the requirements that multiplication be commutative and that there be multiplicative inverses for non-zero elements. The identity element is I_V. L(V) is a non-commutative ring, since in general ST ¹ TS. This is reflected in the fact that matrix multiplication is non-commutative. Only in very special cases is it true that [S][T] = [T][S] (for example, if both [S] and [T] are diagonal matrices, with [S]_ij = 0 for i ¹ j, and [T]_ij = 0 for i ¹ j).

• Note that for operators, it makes sense to talk about T °T, and we can thus define Tⁿ = T °T °¼°T (n times). We also define T⁰ = I, the identity operator.

• Thus, if we have a polynomial P(x) = a_nxⁿ + a_n-1x^n-1 + ¼ + a₁x + a₀ from F[x], we can talk about the polynomial in T:

  P(T) = anTn + an-1Tn-1 + ¼+ a1T + a0.

  (The a₀ term stands for a₀I.) This is an operator on V which acts on vectors as:

  P(T)(v) = anTn (v) + ¼+ a1T(v) + a0v.

• In fact, we can generalize this to power series. If

  p(x) = ¥ å n = 0  anxn,

  then we can define

  p(T) = ¥ å n = 0  anTn.

• This means, for example, that we can define the exponential of an operator:

  exp(T) = ¥ å n = 0  1 n! Tn.

• We can even define the cosine or sine of an operator, etc.:

  cos(T) = ¥ å n = 0  (-1)n (2n)! T2n,

  sin(T) = ¥ å n = 0  (-1)n (2n+1)! T2n+1.

• We can talk about square roots of an operator, saying that S is a square root of T if S² = T. It is unlikely that S is unique.

• We can also talk about logarithms of an operator, saying that S is a logarithm of T if exp(S) = T. Again, it is unlikely that S is unique.

• Examples of linear operators:
  1. A first important example begins like this: suppose we have a system of m linear equations in n variables

     a11x1 + a12x2 + ¼+ a1nxn = b1 a21x1 + a22x2 + ¼+ a2nxn = 21 :            : am1x1 + am2x2 + ¼+ amnxn = bm.

     If we let A : Fⁿ ® F^m be the linear transformation with [A]_ij = a_ij, x be the vector (x₁, x₂, ¼, x_n)^t, and b the vector (b₁, b₂, ¼, b_m)^t, then we can rewrite the equation in the form:

     Ax = b.

     We then have several possibilities.
     1. The first case is when m = n. There are then two possibilities:
        1. If A is an isomorphism, then A^-1 exists, and we can solve the equation for x:

           x = A-1Ax = A-1b.

        2. If A is not an isomorphism, then in particular A is not a an epimorphism, and thus it is possible that b Ï im(A). In this case, there are no solutions to the equation.

           On the other hand, if b Î im(A) there is at least one solution x₀ with Ax₀ = b.

           Note, though, that we also know A is not a monomorphism, and hence dim(ker(A)) ³ 1. Then, if z Î ker(A), we have A(x₀ + z) = Ax₀ + Az = Ax₀ + 0 = Ax₀ = b, and so x₀ + z is another solution. Furthermore, if y is another solution with Ay = b, then A(x₀ - y) = Ax₀ -Ay = b - b = 0. This means x₀ - y Î ker(A), and so y = x₀ + z for some z Î ker(A).

           Thus, if x₀ is a particular solution, then every solution is of the form x₀ + z for some z Î ker(A). The space of solutions is then a translation of the kernel of A, of the form x₀ + ker(A). We then only need to find one particular solution.

           In this case, we have broken the problem down into two parts: first, we solve Ax = 0 (called the homogeneous equation), then we find a single solution Ax₀ = b. For F = R or C, there will be infinitely many solutions.

     2. The second case is when m < n. In this case, A cannot be a monomorphism, and things are then similar to the second part of the previous case. If A is an epimorphism, there is sure to be at least one solution. Again, we solve the homogeneous case, and then find one particular solution. If A is not an epimorphism, there may not be any solutions. Otherwise, as above, in general we will have infinitely many solutions.

     3. If m > n, then A cannot be an epimorphism, and hence there may not be any solutions. If A is a monomorphism, if there is a solution, there will be just one. If A is not a monomorphism, if there is one solution, there will be infinitely many, as above.

     4. In all three cases, we can start by trying to find the inverse (or left partial inverse) of A. If A is a monomorphism, we can expect to be successful, and to be able to find the unique solution if it exists. If A is not a monomorphism, we solve the homogeneous equation, and then look for a single particular solution.

     5. Question: How can we tell if A is a monomorphism? How can we find A^-1, or at least S₁, the left partial inverse of A?

  2. A second important example is the derivative operator

     D: C¥(R) ® C¥(R).

     Note that this is a linear operator, since D(f + g) = D(f) + D(g), and D(af) = aD(f).

     Note that ker(D) = span({1}), the one-dimensional space consisting of all constant functions. If we collapse ker(D) down to nothing (in technical terms, form the quotient space ...) then we can think of D as an isomorphism (on the quotient space). D has an inverse,

     ó õ : C¥(R) ® C¥(R).

     We then have (a variant of) the fundamental theorem of calculus:

     D ó õ (f) = ó õ D(f) = f

     (on the reduced space - that is, up to a constant).

  3. We can then consider the general linear ordinary differential operators with constant coefficients. These are operators in R[D], that is operators of the form:

     P(D) = anDn + ¼+ a1D+ a0.

     These operators act on a function f Î C^¥(R) as

     P(D)(f) = anDn(f) + ¼+ a1D(f) + a0f.

     We then have the general nth order linear ordinary differential equation with constant coefficients:

     P(D)(f) = g.

     We can work on solving these equations with an approach similar to the method for systems of linear equations.

     We first note that if the function f₀ is a solution to this equation, and z = z(x) Î ker(P(D)), then f₀ + z is also a solution, and if f₁ is another solution, then P(D)(f₀ - f₁) = P(D)(f₀) - P(D)(f₁) = g - g = 0. Thus, all solutions are of the form f₀ + z where f₀ is some particular solution, and z Î ker(P(D)).

     We thus separate the problem into two parts. First we solve the associated homogeneous equation:

     P(D)(f) = 0

     to find ker(P(D)), and then we look for a single particular solution to the original equation.

     In general, we have that dim(ker(P(D))) = n, the degree of P. This is not an entirely obvious fact, but it is not counterintuitive ...

     Hence what we need to do is find n functions which form a basis for ker(P(D)). What we need, then, are n linearly independent functions each of which is a solution to the homogeneous equation.

     In theory (:-) this is not too hard.

     We note first that for the first-order case, we have the solution:

     (D- r)(erx) = D(erx) - rerx = rerx - rerx = 0,

     and, in the kth order extension of this,

     (D- r)k(xk-1erx) = 0.

     We also have that the set of functions A = {x^je^r_ix | 0 £ j £ k, 1 £ i £ m}
     is linearly independent. From this we see how to solve equations of the form:

     Õ i  (D- ri)ki(f) = 0.

     Now, consider the operator

     D2 - 2sD+ s2 + t2

     for s, t Î R.

     We have that

     (D2 -2sD+ s2 + t2)k(xk-1esxcos(tx)) = 0,

     (D2 -2sD+ s2 + t2)k(xk-1esxsin(tx)) = 0.

     We can think of this as a = s + ti, and that we are working with

     D2 - 2sD+ s2 + t2 = (D- a)(D- a   ).

     We have that the set of functions

     B = {xjeskxcos(tkx), xmesnxsin(tnx)}

     is also linearly independent. From this we see how to solve equations of the form:

     Õ i  (D2 -2siD+ si2 + ti2)ki(f) = 0.

     We can now put together all the pieces to solve the homogeneous equation P(D) = 0. We use the fact that any polynomial over R can be completely factored as

     P(D) = Õ i  (D- ri)ki Õ j  (D-aj)kj (D- a   j  )kj

     with r_i Î R and a_j Î C.

     To solve the inhomogeneous equation, we need only to find one particular solution of P(D)(f) = g.

     This is just the bare beginnings of techniques for solving differential equations, but it gives the flavor of some relatively powerful methods, and the role that linear algebra plays. I haven't even mentioned the issues of initial values/boundary conditions. For much more on these topics, look at a book such as Elementary Differential Equations with Linear Algebra, by Finney and Ostberg.

  4. These approaches generalize to linear ordinary differential operators with non-constant coefficients:

     an(x)Dn + an-1(x)Dn-1 + ¼+ a0(x),

     to systems of linear ordinary differential operators, and on to linear partial differential operators and systems of linear partial differential operators. But I think for now someone else will have to write that up ... :-)

  5. We can develop a similar approach to solving linear discrete difference equations. For example, the difference equation (a_n+2 - a_n+1 - a_n) = (0) has as a solution the Fibonnacci sequence (a_n) = (1, 1, 2, 3, 5, 8, ¼). We could work with the discrete version D of the differential operator D, where Df(x) = (f(x + 1) - f(x))/1 = f(x + 1) - f(x). In the discrete case, we can't let h go to zero.

     We would then have D((a_n)) = (a_n+1 - a_n). Our example difference equation would be

     (D2 + D-1)((an)) = (0).

     Often, however, it is more convenient to work with the discrete increment operator E, with E(f(x)) = f(x + 1), or E((a_n)) = (a_n+1). Both D and E are linear operators. Our example equation is then E²((a_n)) = E((a_n)) + (a_n), or

     (E2 - E - 1)((an)) = (0).

     In a general form, we have a polynomial P(x) for an equation of the form P(E)((a_n)) = (0). We can then find a general solution by using the facts that

     (E - a)k((njan)) = (0)

     for 0 £ j < k, and

     {(njan) | 0 £ j < k}

     is a linearly independent set, etc.

  6. Of course, all of these are the easy cases. The hard ones are the nonlinear equations ...

  7. At times, this reminds me of a comment made by my Ph.D. thesis advisor, after I had finished the proof of my main result for all odd primes. He said there were three ways to think about it:

     1. I had handled infinitely many cases, and omitted only one, the even prime 2.
     2. I had done half the cases. I had handled all the odd primes, but none of the even ones.
     3. I had dealt with all the uninteresting cases, but not the single interesting case :-)

1. Find a linear operator S such that

   S2 = é ê ë -1 0 0 -1 ù ú û

2. If S is the operator

   S = é ê ê ê ê ê ê ê ê ë 0 - p 4 p 4 0 ù ú ú ú ú ú ú ú ú û ,

   what is exp(S)?

3. Solve the system of equations:

   3x1 + x2 - 2x3 = 3 x1 - 2x2 - x3 = 1 2x1 - 4x2 - x3 = 3

4. Solve the differential equation:

   d5f dx5 - 7 d4f dx4 + 23 d3f dx3 - 45 d2f dx2 + 48 df dx -20f

   = 26cos(x) - 18sin(x)

   Hint:

   x5 - 7x4 + 23x3 - 45x2 + 48x - 20

   = (x - 1)(x - 2)2(x2 - 2x + 5)

5. Find the general solution to the difference equation

   (E2 - E - 1)((an)) = (0)

6. Find the general solution to the difference equation

   (E4 + 2E2 + 1)((an)) = (0)

7. Verify that

   1. D, the derivative, is a linear operator.
   2. (D- r)^k(x^k-1e^rx) = 0

      (D2 -2sD+ s2 + t2)k(xk-1esxcos(tx)) = 0 (D2 -2sD+ s2 + t2)k(xk-1esxsin(tx)) = 0

   3. E, the discrete increment, is a linear operator.
   4. (E - a)^k((n^jaⁿ)) = (0) for 0 £ j < k.

8. What happens in nonlinear cases? Sometimes they are manageable, sometimes not.

   1. Solve the differential equation

      Df = b * f * (1 - f).

   2. What can be said about the difference equation

      E((an)) = (b * an * (1 - an))?

      Note: this is often called the logistics equation.

• A norm on a real or complex vector space V is a function

  || · || : V ® R

  with the properties, for v, v₁, v₂ Î V,

  1. ||v|| ³ 0
  2. ||v|| = 0 if and only if u = 0
  3. ||av|| = |a| ||v||
  4. ||v₁ + v₂|| £ ||v₁|| + ||v₂||.

  A space with such an associated function is called a normed linear space.

• Using the norm, we can define a topology on the space, and can then talk about continuity of functions or operators, limits of sequences, etc.

  I won't go into this much here beyond a few examples, but good places to look are books on Hilbert Spaces and/or functional analysis. There are a few books indicated in the references.

• Using the norm, we can define a metric on V by: d(v₁, v₂) = ||v₁ - v₂||. Metrics have the properties:

  1. d(v₁, v₂) ³ 0
  2. d(v₁, v₂) = 0 iff v₁ = v₂
  3. d(v₁, v₂) = d(v₂, v₁)
  4. d(v₁, v₂) £ d(v₁, v₃) + d(v₃, v₂)
     (the triangle inequality).

• Some examples:
  1. On Rⁿ or Cⁿ, we have the norm

     ||(a1, a2, ¼, an)||2 = (|a1|2 + ¼+ |an|2)1/2.

     This is usually called the Euclidean norm.

     We can also think of this in terms of the inner product given by
     < (a₁, a₂, ¼, a_n), (b₁, b₂, ¼, b_n) >
           = a₁[`b]<sub>1</sub> + a<sub>2</sub>[`b]₂ + ... + a_n[`b]_n.

     We then have ||v||₂ = < v, v > ^1/2.

  2. We can generalize, for p > 0, to

     ||(a1, a2, ¼, an)||p = (|a1|p + ¼+ |an|p)1/p,

     and

     ||(a1, a2, ¼, an)||¥ = max (|a1|, ¼, |an|).

     These are called the p-norms and the ¥-norm (or max-norm).

     A fun little exercise is to draw the circle of radius 1 in R² for each of these norms:

     circlep(1) = {(x, y) Î R2 | ||(x, y)||p = 1}

     for p > 0, or for p = ¥.

     One of these constitutes a ``proof'' that a square is a circle :-)

  3. We can generalize this to R^¥ or C^¥:

     ||(a1, a2, ¼)||p = æ è ¥ å n = 1  |an|p ö ø 1/p   , ||(a1, a2, ¼)||¥ = sup n  (|an|).

     For this to make sense, we will have to limit ourselves to the subspaces where the sum converges:

     lp = {(a1, a2, ¼) |  æ è ¥ å n = 1  |an|p ö ø 1/p   < ¥}, l¥ = {(a1, a2, ¼) |  sup n  (|an|) < ¥}.

  4. On C⁰(R) or C⁰(C), we can define

     ||f||p = æ è ó õ |f|p ö ø 1/p

     ||f||¥ = sup x  (|f(x)|).

     Again, we limit ourselves to the subspaces where these are < ¥, and call the corresponding spaces L^p or L^¥.

  5. We can think each of these Euclidean norms, || · ||₂, as coming from an inner product:

     ||(a1, a2, ¼)||2 = < (a1, ¼), (a1, ¼) > 1/2 = æ è ¥ å n = 1  an _ a   n  ö ø 1/2   ||f||2 = < f, f > 1/2 = æ è ó õ f _ f   ö ø 1/2   .

     When a complex Euclidean normed linear space is complete (that is, every Cauchy sequence converges), it is called a Hilbert space.

1. Verify that each of the examples actually are norms.

2. In R², draw the unit circles
   1. circle₁(1) = {(x, y) Î R² | ||(x, y)||₁ = 1}
   2. circle_3/2(1) = {(x, y) Î R² |                 
      ||(x, y)||_3/2 = 1}
   3. circle₂(1) = {(x, y) Î R² | ||(x, y)||₂ = 1}
   4. circle₃(1) = {(x, y) Î R² | ||(x, y)||₃ = 1}
   5. circle_¥(1) = {(x, y) Î R² |                 
      ||(x, y)||_¥ = 1}

3. Suppose that V is a real or complex vector space. An inner product on V is a conjugate-bilinear function on V:

   < ·, · > : V x V ® F

   where, for all v₁, v₂, v Î V, and a Î F,

   < v , v > ³ 0 < v , v > = 0 iff v = 0 < v1 + v2 , v > = < v1 , v > + < v2 , v > < v1 , v2 > = < v2 , v1 >   < av1 , v2 > = a < v1 , v2 > .

   Show that for an inner product,

   < v , v1 + v2 > = < v , v1 > + < v , v2 > < v1 , av2 > = a   < v1 , v2 > .

   Show that for a finite dimensional real or complex vector space V with basis (v₁, v₂, ¼, v_n), the function f : V x V ® F given by

   f æ è å i  ai vi, å i  bi vi ö ø = å i  ai bi

   is an inner product.
4. Recall that a linear functional on a vector space V is a linear map f : V ® F. For a finite dimensional real or complex inner product space V, define the dual space of V to be the space

   V* = { f : V ® F |  f is a linear functional}

   Show that V^* is a vector space over F, and that V and V^* are isomorphic to each other.

   Hint: Show that every f Î V^* corresponds with a function of the form

   < vf , · > : V ® F

   for some v_f Î V.

• Suppose T is a linear operator on V. Then l Î F is called an eigenvalue or characteristic value of T if there exists v ¹ 0 in V with T(v) = lv. In this case we call v an eigenvector or characteristic vector of T.

• An equivalent definition is that l is an eigenvalue of T if and only if (T - lI)v = 0 for some v ¹ 0. This follows from the fact that 0 = (T - lI)v = T(v) - lv if and only if T(v) = lv. Note that this also means that ker(T - lI) ¹ {0}. Thus, l is an eigenvalue if and only if (T - lI) is not a monomorphism. In the finite dimensional case, this is equivalent to (T - lI) not being invertible.

• Now suppose that l₁, ..., l_k are distinct eigenvalues of T (i.e., l_i ¹ l_j for i ¹ j), with associated eigenvectors v₁, ¼, v_k. Then {v₁, ¼, v_k} is a linearly independent set.

• This means in particular that if T Î L(V) with dim(V) = n, and T has n distinct eigenvalues l_i with associated eigenvectors v_i, then the set S = (v₁, ¼, v_n) is a basis for V. Furthermore, the matrix representation of T with respect to the basis S is

  [T] = é ê ê ê ê ë l1 0 ¼ 0 0 l2 ¼ 0 : : : : 0 0 ¼ ln ù ú ú ú ú û

  or [T]_ii = l_i, [T]_ij = 0 for i ¹ j. This is also sometimes written as [T] = diag(l₁, ¼, l_n).

  Eigenvalues and eigenvectors can thus give us a very simple representation for T.

• Not all linear operators have eigenvalues (or eigenvectors). For example, the linear operator T_q : R² ® R² given by

  [Tq] = é ê ë cos(q) -sin(q) sin(q) cos(q) ù ú û

  is a (counter-clockwise) rotation around the origin through the angle q. If q is not an integral multiple of p, then T_q has no eigenvalues.

• More generally, consider a non-zero linear operator T : R² ® R². If 0 ¹ u Î R², then there must be a linear combination of {u, T(u), T²(u)} with

  0 = T2(u) + aT(u) + b(u) = Pu(T)(u).

  If P_u(T) factors into linear factors P_u(T) = (T - l₁I)(T - l₂I) with l₁, l₂ Î R, then l₁ and l₂ are eigenvalues of T. On the other hand, if a² - 4b < 0, then T has no eigenvalues (it is a general rotation in R²).

• On the other hand, every non-zero linear operator T : Cⁿ ® Cⁿ has an eigenvalue. We can see this from the fact that, for any u ¹ 0 in Cⁿ, the set of vectors {u, T(u), T²(u), ¼, Tⁿ(u)} must be a linearly dependent set, and hence there are a₀, a₁, ¼, a_n not all zero with

  0 = n å i = 0  aiTi(u) = Pu(T)(u).

  We know that we can factor the polynomial P_u(T) over C into a product of linear factors

  Pu(T) = Õ j  (T - zjI)kj(T).

  This linear operator is not a monomorphism (since u Î ker(P_u(T))). It is a product of commutative factors, and hence at least one of the factors (T - z_jI) is not a monomorphism. This says then that at least one of the z_j is an eigenvalue of T. We can proceed to find n eigenvalues and eigenvectors.

• For for every non-zero linear operator T : Cⁿ ® Cⁿ, there is a polynomial P_T(T) = å_ia_iTⁱ of degree n, called the characteristic polynomial of T, with:

  1. P_T(T) = 0. That is, P_T(T)(u) = 0 for all u Î Cⁿ.
  2. The linear factors of P_T(T) are each of the form (T - l_jI) for some eigenvalue l_j. Also, a_n = 1. That is,

     PT(T) = Õ j  (T - ljI)kj(T).

  3. For each eigenvalue l_j, there is an eigenvector u_j. If k_j > 1, there is a linearly independent set of k_j eigenvectors for l_j. The set of n eigenvectors {u_i} is a linearly independent set, and thus is a basis for Cⁿ. As above, the matrix representation of T with respect to this basis is [T] = diag(l₁, ¼, l_n) (with repetitions as necessary).

• A non-zero linear operator T : Rⁿ ® Rⁿ also has a characteristic polynomial of degree n. The big difference for the real case is that the polynomial will factor into a combination of linear and quadratic factors. There is then a basis consisting partially of eigenvectors and partially of pairs of basis vectors for two-dimensional subspaces on which T is a rotation. The matrix of T with respect to this basis then looks like [T] = diag(l₁, ¼, l_k, A_k+1,¼,A_(n-k)/2) where each A_i is a two-by-two matrix without eigenvalues.

• The set of eigenvalues of an operator is sometimes called the spectrum of the operator. In cases like Cⁿ where the eigenvectors form a basis for the space, using such as basis is sometimes known as the spectral decomposition of the operator.

• Just briefly, let's glance at differential operators. D: C^¥(R) ® C^¥(R) has uncountably many eigenvalues. Each real number a Î R is an eigenvalue since

  D(eax) = aeax.

  The corresponding eigenvectors are of course f_a(x) = e^ax.

  The differential operator D² also has uncountably many eigenvalues and eigenvectors since for a > 0,

  D2(cos(Öax)) = -acos(Öax), D2(sin(Öax)) = -asin(Öax), and D2(eÖax) = aeÖax.

• In this context, here's an example which puts many of these pieces together. In quantum mechanics, a typical expression of Schrödinger's equation looks like

  é ê ë - (h/2p)2 2me Del2+V(x,y,z) - i(h/2p) ¶ ¶t ù ú û Y = 0.

  This example is for an electron (with mass m_e) in a potential field V(x,y,z).

  The general solution of this operator equation is

  Y(x,y,z,t) = ¥ å n = 0  cnYn(x,y,z)exp æ ç è -iEnt (h/2p) ö ÷ ø

  where Y_n(x,y,z) is an eigenfunction solution of the associated time independent Schrödinger equation, with E_n the corresponding eigenvalue. The inner product, giving a time dependent probability, looks like

  P(t) = ó õ Y Y   dv.

1. Show that if v is an eigenvector of T corresponding with the eigenvalue l, and a Î F (a ¹ 0), then av is also an eigenvector of T corresponding with l. In particular, eigenvectors are not unique.

2. Define T Î L(F²) by T((x, y)) = (y, x). Find all the eigenvalues and eigenvectors of T.

3. Define T Î L(F³) by T((x, y, z)) = (-y, 0, 2z). Find all the eigenvalues and eigenvectors of T.

4. Define T Î L(F^¥) by T((a₁, a₂, ¼)) = (a₂, a₃, ¼) (i.e., T is the left shift operator). Find all the eigenvalues and eigenvectors of T.

5. Suppose T Î L(V) is invertible. Show that l ¹ 0 is an eigenvalue of T if and only if l^-1 is an eigenvalue of T^-1.

6. Suppose S, T Î L(V). Show that ST and TS have the same eigenvalues.

7. Give an example of an operator whose matrix has all zeros on the diagonal with respect to some basis, but which is invertible. Give an example of an operator whose matrix has all diagonal elements non-zero with respect to some basis, but which is singular (i.e., has no inverse).

8. Show that if S, T Î L(V), S is invertible, and P(x) Î F[x], then P(S^-1TS) = S^-1P(T)S.

9. Suppose that V is a finite dimensional normed complex vector space, and T is an isometry of V (i.e., ||T(v)|| = ||v|| for all v Î V). Show that every eigenvalue l of T has |l| = 1. Hint: show that there is a basis for V consisting of eigenvectors e_i, with ||e_i|| = 1 for 1 £ i £ n.

10. Let V be a complex vector space, and T Î L(V). Let P(x) Î C[x]. Show that a Î C is an eigenvalue of P(T) if and only if a = P(l) for some eigenvalue l of T.

11. Is the preceding true for real vector spaces? Why not? (Note: this is another example of why C is so much nicer a place to work than R ...)

    Is the preceding true if we replace C[x] with C^¥[x], power series? If so, show it. If not, give a counterexample.

• Recall that the matrix associated with a linear transformation depends on the particular bases we use. In the case of a linear operator T : V ® V, it is typical to use the same base for V as domain and as codomain. However, as we have seen, a basis consisting of eigenvectors is particularly convenient, and hence it is useful to know how to change bases.

  If V is finite dimensional, with two bases S₁ = (u₁, ¼, u_n) and S₂ = (v₁, ¼, v_n), we can consider the matrices of T with respect to the mixed bases. We can indicate this by the various symbols [T]^S₁, [T]^S₂, [T]^S₁S₂, and [T]^S₂S₁, where, for example

  T(ui) = å j  [T]jiS1uj, and T(ui) = å j  [T]jiS1S2vj.

• We can look in particular at the matrix representations for the identity operator I. For ease of notation, let [B] = [I]^S₁S₂, the matrix representation for I using S₁ for the domain and S₂ for the codomain. In particular, we then have

  ui = å j  [B]jivj = Bijvj.

  We also have that the matrix inverse [B]^-1 = [I]^S₂S₁ which takes us in the opposite direction. Putting the pieces together, we then have

  [B][T]S1 = [T]S2[B].

  That is, doing the operator T in the basis S₁ and then converting to the basis S₂ is the same as converting bases first, and then doing T in the second base.

  Another way to say this is:

  [T]S1 = [B]-1[T]S2[B]

  or

  [T]S2 = [B][T]S1[B]-1.

1. Show that if A and B are operators on a finite dimensional vector space with AB = I, the identity operator, then BA = I, and so B = A^-1.

2. Suppose that T is an operator that has the same matrix with respect to every basis. Show that T must be some multiple of the identity operator I.

• Let V be a finite dimensional vector space over R or C, and T : V ® V an operator on V. Let P_T(T) = å_{i = 0}ⁿ a_iTⁱ be the characteristic polynomial of T. We can then define the trace of T by trace(T) = a_n-1, the coefficient of T^n-1.

  We can also define the trace of an n x n real or complex matrix A by trace_m(A) = å_{i = 1}ⁿA_ii, the sum of the diagonal elements of A.

  These definitions are consistent, in the sense that trace(T) = trace_m([T]), where [T] is the matrix of T with respect to some basis for V. An exercise will be to show that it doesn't matter what basis we use (they all come out the same). Since these definitions are consistent, we will ordinarily write them both the same way, as trace().

• For S,T : V ® V, and a, b Î F, the trace has the nice properties:

  1. trace(aS + bT) = a trace(T) + b trace(T). This says that the trace is a linear functional on L(V).
  2. In the complex case, we have

     trace(T) = å i  li

     where the l_i are the eigenvalues of T.
  3. trace(ST) = trace(TS).

• From these we can derive nice facts like:

  If we define the commutator of S and T by [S, T] = ST - TS, then we always have [S, T] ¹ I, the identity operator.

• We can define the determinant of an operator T by det(T) = (-1)ⁿ a₀, where a₀ is the constant term in the characteristic polynomial.

• The determinant has the nice properties:

  1. In the complex case, we have

     det (T) = Õ i  li

     where the l_i are the eigenvalues of T.
  2. det(T) ¹ 0 iff ker(T) = {0} iff T is invertible.
  3. det(ST) = det(S)det(T) = det(TS).
  4. P_T(z) = det(zI - T). That is, the characteristic polynomial is the determinant of the generalized operator (zI -T), where we think of z as a complex variable.
  5. We can define the determinant of a square matrix M = [a_ij] by

     detm(M) = å p Î perm(n)  sign(p) Õ i  ap(i)i

     where perm(n) is the set of all permutations of the numbers (1, 2, ¼, n), sign(p) is the sign of the permutation, and p(i) is the ith number in the permutation p.

     We then have:

     detm([T]) = det (T).

     As for the trace, since the two definitions are consistent, we will typically denote both determinants by the same symbol det().

     Showing that these two definitions are consistent is a fair amount of work (just looking at the formula for det_m() should give you some idea). You can look it up if you are interested.

  6. There is another definition of determinant. Suppose V is a real or complex n-dimensional vector space. Then there is an alternating multi-linear functional

     detv: Vn ® F

     with the properties, for all v₁, v₂, ¼, v_n, v_i1, v_i2 Î V and a, b Î F:

     1. det_v(v₁, ¼, v_i, ¼, v_j, ¼, v_n) = - det_v(v₁, ¼, v_j, ¼, v_i, ¼, v_n) (alternating)

     2. det_v(v₁, ¼, av_i1 + bv_i2, ¼, v_n) = adet_v(v₁, ¼, v_i1, ¼, v_n) + bdet_v(v₁, ¼, v_i2, ¼, v_n) (multi-linear)

     3. det_v(e₁, ¼, e_n) = 1 for a particular basis {e₁, ¼, e_n} for V. (uniqueness)

     We then have det_m(A) = det_v(A₁, ¼, A_n), where A_k is the kth column of the matrix A, considered as a vector.

     We have that det(T) = det_m([T]) = det_v([T]₁, ¼, [T]_n). The fact that there are three different versions of the same thing suggests that many people have worked on this topic, and that this topic occurs in a variety of contexts ...

  7. Recall the discussion of norms on vector spaces. The Euclidean norms (|| v ||₂), which can be thought of as coming from an inner product ( < v₁, v₂ > ), have a nice relationship with the determinant:

     ||T(v)||2 = || v ||2 for all v Î U

     iff | det (T)| = 1.

  8. An operator which preserves the Euclidean norm (||T(v)||₂ = ||v||₂) is called an isometry. When the scalar field is R, these are called orthogonal operators. In the case of C, they are called unitary operators. In the case of the quaternions, they are called symplectic operators.

  9. The isometries on a normed linear space have the properties:

     1. If T is an isometry, then det(T) ¹ 0, and hence T has an inverse, T^-1, which is also an isometry. Of course, I, the identity operator, is an isometry.
     2. If T₁ and T₂ are isometries, then T₁T₂ is also an isometry.
     3. This means that the set of isometries forms a group. In the three cases described above, these are called the orthogonal group, the unitary group, and the symplectic group.

  10. Another way to characterize the isometries (in the finite dimensional case, but also with appropriate generalizations in infinite dimensional cases) is that an operator is an isometry if 1.) its inverse exists, and 2.) the matrix of its inverse is given by the conjugate transpose. That is, if

      [T-1]ij = [T]ji   .

  11. The determinant also plays an important role in change of variables formulas (in Rⁿ, for example).

      Suppose W Ì Rⁿ and s: W ® Rⁿ. We will think of s as a (local) coordinate system on W, or as a change of variables.

      The derivative of s at x is the unique operator T (if it exists) satisfying:

      lim y® 0  ||s(x + y) - s(x) - Ty|| ||y|| = 0.

      If this operator T exists, s is called differentiable, and T is denoted by s¢(x). Note that s¢(x) Î L(Rⁿ). We can write s(x) = (s₁(x), ¼, s_n(x)), and we can denote the partial derivative of s_j with respect to the kth coordinate by D_ks_j(x).

      If s is differentiable at x, then the matrix of s¢(x) is given by:

      [s¢(x)] = é ê ê ê ë D1s1(x) ¼ Dns1(x) : : D1sn(x) ¼ Dnsn(x) ù ú ú ú û

      If we assume that W is a reasonable set (e.g., open, or measurable) and f : W® R is also reasonable (e.g., continuous, or measurable), then we have the change of variables formula:

      ó õ s(W)  f(y)dy = ó õ W  f(s(x))| det (s¢(x))|dx.