Eigenvalues and Eigenvectors

We will now review some ideas from linear algebra. Proofs of the theorems are either left as exercises or can be found in any standard text on linear algebra. We know how to solve n linear equations in n unknowns. It was assumed that the determinant of the matrix was nonzero and hence that the solution was unique. In the case of a homogeneous system AX = 0, if , the unique solution is the trivial solution X = 0. If , there exist nontrivial solutions to AX = 0. Suppose that , and consider solutions to the homogeneous linear
system

A homogeneous system of equations always has the trivial solution . Gaussian elimination can be used to obtain the reduced row echelon form which will be used to form a set of relationships between the variables, and a non-trivial solution.

Definition (Linearly Independent):

The vectors are said to be linearly independent if the equation



implies that . If the vectors are not linearly independent they are said to be linearly dependent.

Two vectors in are linearly independent if and only if they are not parallel. Three vectors in are linearly independent if and only if they do not lie in the same plane.

Definition (Linearly Dependent):

The vectors are said to be linearly dependent if there exists a set of numbers not all zero, such that

.

Theorem:

The vectors are linearly dependent if and only if at least one of them is a linear combination of the others.

A desirable feature for a vector space is the ability to express each vector as s linear combination of vectors chosen from a small subset of vectors. This motivates the next definition.

Definition (Basis):

Suppose that is a set of m vectors in . The set S i s called a basis for if for every vector there exists a unique set
of scalars so that X can be expressed as the linear combination

Theorem:

In , any set of n linearly independent vectors forms a basis of . Each vector is uniquely expressed as a linear combination of the basis vectors.

Theorem:

Let be vectors in .

(i) If m>n, then the vectors are linearly independent.

(ii) If m=n, then the vectors are linearly dependent if and only if , where .

Applications of mathematics sometimes encounter the following questions: What are the singularities of , where is a parameter? What is the behavior of the sequence of vectors ? What are the geometric features of a linear transformation? Solutions for problems in many different disciplines, such as economics, engineering, and physics, can involve ideas related to these equations. The theory of eigenvalues and eigenvectors is powerful enough to help solve these otherwise intractable problems.

Let A be a square matrix of dimension n × n and let X be a vector of dimension n. The product Y = AX can be viewed as a linear transformation from n-dimensional space into itself. We want to find scalars for which there exists a nonzero vector X such that

(1) ;

that is, the linear transformation T(X) = AX maps X onto the multiple . When this occurs, we call X an eigenvector that corresponds to the eigenvalue , and together they form the eigenpair for A. In general, the scalar and vector X can involve complex numbers. For simplicity, most of our illustrations will involve real calculations. However, the techniques are easily extended to the complex case. The n × n identity matrix I can be used to write equation (1) in the form

(2) .

The significance of equation (2) is that the product of the matrix and the nonzero vector X is the zero vector! The theorem of homogeneous linear system says that (2) has nontrivial solutions if and only if the matrix is singular, that is,

(3) .

This determinant can be written in the form

(4)

Definition (Characteristic Polynomial):

When the determinant in (4) is expanded, it becomes a polynomial of degree n, which is called the characteristic polynomial

(5)

Exploration For p()

There exist exactly n roots (not necessarily distinct) of a polynomial of degree n. Each root can be substituted into equation (3) to obtain an underdetermined system of equations that has a corresponding nontrivial solution vector X. If is real, a real eigenvector X can be constructed. For emphasis, we state the following definitions.

Definition (Eigenvalue):

If A is and n × n real matrix, then its n eigenvalues are the real and complex roots of the characteristic polynomial

.

Definition (Eigenvector):

If is an eigenvalue of A and the nonzero vector V has the property that



then V is called an eigenvector of A corresponding to the eigenvalue . Together, this eigenvalue and eigenvector V is called an eigenpair .

The characteristic polynomial can be factored in the form



where is called the multiplicity of the eigenvalue . The sum of the multiplicities of all eigenvalues is n; that is,

.

The next three results concern the existence of eigenvectors.

Theorem (Corresponding Eigenvectors):

Suppose that A is and n × n square matrix.
(a) For each distinct eigenvalue there exists at least one eigenvector V corresponding to .
(b) If has multiplicity r, then there exist at most r linearly independent eigenvectors that correspond to .

Theorem (Linearly Independent Eigenvectors):

Suppose that A is and n × n square matrix. If the eigenvalues are distinct and are the k eigenpairs, then is a set of k linearly independent vectors.

Theorem (Complete Set of Eigenvectors):

Suppose that A is and n × n square matrix. If the eigenvalues of A are all distinct, then there exist n nearly independent eigenvectors .

Finding eigenpairs by hand computations is usually done in the following manner. The eigenvalue of multiplicity r is substituted into the equation

.

Then Gaussian elimination can be performed to obtain the row reduced echelon form, which will involve n-k equations in n unknowns, where . Hence there are k free variables to choose. The free variables can be selected in a judicious manner to produce k linearly independent solution vectors that correspond to .