Eigen Structure and Diagonalization

Definition: Let $A$ be an $n \times n$ matrix. The number $\lambda$ is called an eigenvalue of $A$ if there exists an $n \times 1$ nonzero vector $x$ such that $Ax = \lambda x$. Every nonzero vector satisfying $Ax = \lambda x$ is called an eigenvector of $A$ corresponding to the eigenvalue $\lambda$, and $(\lambda,x)$ is called and an eigenpair of $A$. The set of all eigenvalues of $A$ is called the spectrum of $A$, denoted $\sigma(A)$.

Remarks:

1. Eigenvalues are also called proper values (eigen is a German word means proper) or characteristic values or latent values. Eigenvectors are also called proper vectors or characteristic vectors or latent vectors.

2. $Ax = \lambda x$ iff $(A - \lambda I)x = 0$. Thus, $\lambda$ is an eigenvalue of $A$ iff $(A - \lambda I)x = 0$ has a nontrivial solution (i.e. the solution space is not just the zero vector) iff $A - \lambda I$ is singular iff $\det(A - \lambda I) = 0$.

3. Eigenvectors are also called right eigenvectors. Left eigenvectors are defined as follows: $x^T$ is said to be a left eigenvector of $A$ iff $x^T A = \lambda x^T$ (i.e. $x$ is a right eigenvector of $A^T$ associated with the eigenvelue $\lambda$). Note that $x$ here is a column vector. Note also that $A$ and $A^T$ have the same eigenvalues but not necessarily the same eigenvectors.

4. If the eigenvalues of $A$ are distinct, and $x$ is a right eigenvector of $A$ corresponding to the eigenvalue $\lambda$ and $y^T$ is a left eigenvector of $A$ corresponding to the eigenvalue $\mu$, where $\mu \neq \lambda$, then $y^T x = 0$; if $\mu = \lambda$, then $y^T x \neq 0$.

Definition: The polynomial $f(\lambda) = \det(A - \lambda I)$ is called the characteristic polynomial of $A$ and the equation $\det(A - \lambda I) = 0$ is called the characteristic equation of $A$. The roots (zeros) of the characteristic polynomial are the eigenvalues of $A$.

Remarks:

1. Some people call $g(\lambda) = \det(\lambda I - A)$ the characetreristic polynomial and $\det(\lambda I - A) = 0$ the characteristic equation. Note that the characteristic polynomial is of degree $n$ and $g(\lambda) = (-1)^n f(\lambda)$. Thus, $f(\lambda)$ and $g(\lambda)$ have the same roots.

2. If you expand $f(\lambda) = \det(A - \lambda I)$, then
   1. The coefficient of $\lambda^n$ is $(-1)^n$ and the coefficient of $\lambda^{n-1}$ is $(-1)^{n-1} \sum_{i=1}^n a_{ii}$.

   2. $\det(A) = \prod_{i=1}^n \lambda_i = f(0)$, where $\lambda_1$, $\lambda_2$, $\cdots$, $\lambda_n$ are the eigenvalues of $A$ (including repeated eigenvalues).

   3. $\sum_{i=1}^n \lambda_i = \operatorname{tr}(A) = \sum_{i=1}^n a_{ii}$.

   Thus, $\lambda = 0$ is an eigenvalue of $A$ iff $A$ is singular iff $\det(A)=0$ iff $f(0) = 0$.

3. If you expand $g(\lambda)$, then you get similar things (but remember $g(\lambda) = (-1)^n f(\lambda))$. In particular, you'll have here also $\det(A) = \prod_{i=1}^n \lambda_i$ and $\operatorname{tr}(A) = \sum_{i=1}^n \lambda_i$. Thus, the determinant of $A$ is equal to the product of its eigenvalues and the trace of $A$ is equal to the sum of its eigenvalues.

Definition: Let $\lambda$ be an eigenvalue of matrix $A$ and let $W$ be the set consisting of the zero vector and all vectors of $A$ associated with $\lambda$. Then $W$ is a subspace of $\mathbb{C}^n$ and it's called the eigenspace of $A$ associated with $\lambda$. The dimension of $W$ is called the geometric multiplicity of $\lambda$. The algebraic multiplicity of $\lambda$ is the multiplicity of $\lambda$ as a root of $\det(A - \lambda I) = 0$. An eigenvalue is called simple of its algebraic multiplicity is 1, and multiple if its algebraic multiplicity is greater than 1. A matrix $A$ is called stable if the real part of each eigenvalue of $A$ is negative (i.e. all the eigenvalues of $A$ lie in the open left half plane).

Theorem: The geometric multiplicity of an eigenvalue is less than or equal to its algebraic multiplicity.

bf Examples:

1. Let $A=\left[ \begin{array}{cc} 1 & 2 \ 0 & 1\end{array}\right]$. Then $\lambda=1$ is an eigenvalue of $A$ of algebraic multiplicity 2 and a geometric multiplicity 1. Thus, the geometric multiplicity of $\lambda=1$ as an eigenvalue of $A$ is less than its algebraic multiplicity.

2. Let $A=\left[ \begin{array}{ccc} 3 & -1 & -2 \ 2 & 0 & -2 \ 2 & -1 & -1\end{array}\right]$. Then $\lambda=1$ is an eigenvalue of $A$ of algebraic multiplicity 2 and a geometric multiplicity 2. Thus, the geometric multiplicity of $\lambda=1$ as an eigenvalue of $A$ is equal to its algebraic multiplicity.

Definition: An $n \times n$ matrix is called diagonalizable (or cane be diagonalized) iff there exists a nonsingular matrix $P$ and a diagonal matrix $D$ such that $P^{-1} A P = D$. If such a matrix $P$ exists, we say $P$ diagonalizes $A$. I.e. $A$ is diagonalizable iff it's similar to a diagonal matrix.

Definition: If an $n \times n$ matrix has less than $n$ linearly independent eigenvectors, $A$ is called defective.

Definition: Let $L$ be a linear operator on a vector space $V$. An eigenvector of $L$ is a nonzero vector $x$ in $V$ such that $L(x) = \lambda x$ for some scalar $\lambda$. In this case, we say $\lambda$ is an eigenvalue of $L$.

Remark: Let $A$ be the matrix of a linear operator $L$ (i.e. $L(x) = Ax$). Then the eigenvalues/eigenvectors of $L$ and $A$ are the same.

Note: The problem in which we have to determine eigenvalues/eigenvectors is called an eigenvalue problem.

How to find the eigenvalues and associated eigenvectors of an $n \times n$ matrix $A$?

1. Eigenvalues: find the roots of $\det(A - \lambda I)$; i.e. solve the equation $\det(A - \lambda I) = 0$. These are the eigenvalues of $A$.

2. Eigenvectors: For each eigenvalue $\lambda$, find a basis for the solution space of $(A - \lambda I)x = 0$. The vectors in the basis are linearly independent eigenvectors of $A$ associated with $\lambda$.

Reminder of definitions we introduced in the past:

1. An $n \times n$ matrix is called nilpotent iff $A^k =0$ for some positive integer $k$.

2. An $n \times n$ matrix is called idempotent iff $A^2 = A$.

3. An $n \times n$ matrix $B$ is said to be similar to the $n \times n$ matrix $A$ iff there exists an invertible $n \times n$ matrix $P$ such that $B = P^{-1}AP$. Note that similarity is an equivalence relation.

Remark: Let $x_1, x_2, \cdots, x_n$ be eigenvectors of the $n \times n$ matrix $A$ corresponding to the eigfenvalues $\lambda_1, \lambda_2, \cdots, \lambda_n$, and let $T=[x_1,x_2, \cdots,x_n]$. Then $AT = [\lambda_1 x_1,\lambda_2 x_2, \cdots, \lambda_n x_n] = T D$, where $D=\operatorname{diag}(\lambda_1, \lambda_2, \cdots, \lambda_n)$. If $x_1, x_2, \cdots, x_n$ are linearly independent, then $T$ is nonsingular, and thus we get, $T^{-1}AT=D$, which means $T$ diagonalizes $A$. Note also we can write $T^{-1}A = D T^{-1}$. Thus, the columns of $T$ are right eigenvectors of $A$ and the rows of $T^{-1}$ are left eigenvectors of $A$ (and right eigenvectors of $A^T$).

Theorems: Let $A$ and $B$ be $n \times n$ matrices.

1. If $(\lambda,x)$ is an eigenpair of $A$ and $r$ is a nonzero number, then $\lambda,rx)$ is an eigenpair of $A$, and $(\lambda^k,x)$ is an eigenpair of $A^k$, $\forall k \in \mathbb{Z}^+$. If $\lambda$ is a non-real eigenvalue and $A$ is real, then $(\overline{\lambda},\overline{x})$ is an eigenpair of $A$.

2. $A$ and $A^T$ have the same eigenvalues but not necessarily the same eigenvectors.

3. $AB$ and $BA$ have the same eigenvalues but not necessarily the same eigenvectors.

4. If $A$ is nonsingular, then $(\lambda,x)$ is an eigenpair of $A$ iff $(\frac{1}{\lambda},x)$ is an eigenpair of $A^{-1}$.

5. The eigenvalues of a diagonal/lower-triangular/upper-triangular matrix are equal to the elements of the main diagonal.

6. $A$ is diagonalizable iff $A$ has $n$ linearly independent eigenvectors. Moreover, if $A$ is similar to a diagonal matrix $D$, then the elements of $D$ are the eigenvalues of $A$.

7. If $A$ is defective, then $A$ is non-diagonalizable.

8. If $A$ is nondefective and $x_1, x_2,  \cdots, x_n$ are linearly independent eigenvectors of $A$ corresponding to the eigenvalues $\lambda_1$, $\lambda_2$, $\cdots$, $\lambda_n$ respectively, and $P = [x_1,x_2, \cdots,x_n$], then $P^{-1}AP = \operatorname{diag}(\lambda_1, \lambda_2, \cdots, \lambda_n)$.

9. If $A$ and $B$ are similar, then they have the same eigenvalues. You can easily find a relationship between their eigenvectors.

10. If $A$ is idempotent and $\lambda$ is an eigenvalue of $A$, then $\lambda = 0$ or $\lambda=1$.

11. If $A$ is nilpotent and $\lambda$ is an eigenvalue of $A$, then $\lambda = 0$.

12. The eigenvalues of a Hermitian matrix are real and the eigenvalues of a skew-Hermitian matrix have zero real parts.

13. If $\lambda$ is an eigenvalue of a unitary matrix, then $\vert\lambda\vert = 1$.

14. $A$ is normal iff $A$ has $n$ orthogonal eigenvectors.

15. For a $2 \times 2$ matrix, the characteristic polynomial is $\lambda^2 - \operatorname{tr}(A) \lambda + \det(A) = 0$.

16. If $A$ is diagonal, then so are $A^T$ and $A^k$, $\forall k \in \mathbb{Z}^+$, and if $A$ is nonsingular, then $A^{-1}$ is also diagonalizable.

17. If the eigenvalues of $A$ are distinct, then $A$ is diagonalizable.

18. $A$ is diagonalizable iff $A$ has a complete set of $n$ linearly independent eigenvectors.

Exercises:

Prove (1), (2), (4), (5), (7), (9) (here also find the relationship between the eigenvetors of $A$ and those of $B$), (10), (11), (12), (13), (15), (16).

Click here for the next part

REFERENCES: