Eigenvalues and Eigenvectors

Table of Contents

• Invariant Subspace
• Eigenvalues
  • Multiplicities of Eigenvalues
  • Distinct eigenvalues
• Diagonalization
  • Powers of a diagonalizable matrix

If for some vector, the output of a linear transformation is just a scaled multiple of the same vector without any changes in the direction, we call such a vector an “invariant” direction or an eigenvector and the scalar the eigenvalue.

$$\begin{array}{rlrl}& & Au& =\lambda u\\ & \Rightarrow & Au-\lambda u& =0\\ & \Rightarrow & (A-\lambda I)u& =0\end{array}$$

Since $u$ is a non-zero vector, we are looking for a solution to the homogeneous system of equation $A-\lambda I=0$. Thus $A-\lambda I$ is a singular matrix, and consequently $|A-\lambda I|=0$.

Invariant Subspace

If some vector $t$ is an eigenvector for some matrix $A$ such that $At=\lambda t$, then all scalar multiples of $t$ are eigenvectors as well.

Because the space of all multiples of $t$ would pass through the origin if $k=0$ in $kt$, these eigenvectors form a subspace. Such a subspace is known as an invariant subspace.

$\underline{\text{Definition}}-$ The invariant subspace of a linear transformation $T:V_1\to V_2$ is the set of vectors $u$ such that $T(u)=\lambda u$ where $\lambda$ is a scalar.

Eigenvalues

$\underline{\text{Definition}}-$ The scaling factor by which eigenvectors get scaled upon being transformed are called eigenvalues.

The polynomial equation obtained while doing $\det (A-\lambda I)=0$ is called the characteristic polynomial. The roots of this polynomial are the eigenvalues. Due to this, eigenvalues can be-

1. Real and distinct
2. Real but repeated
3. Complex

Multiplicities of Eigenvalues

1. Algebraic Multiplicity (AM )- The number of times an eigenvalue is repeated.
2. Geometric Multiplicity (GM) - The number of linearly independent eigenvectors associated with a particular eigenvalue.
3. Note - GM $\le$ AM for each eigenvalue $\lambda$. If GM $<$ AM for any specific eigenvalue, we say that the corresponding eigenvalue is deficient.

Distinct eigenvalues

If the eigenvalues of a transformation are distinct, the eigenvectors are linearly independent.

$\underline{\text{Proof}}-$ Consider a linear transformation on ${\mathbb{R}}^2$ which has $u_1$ and $u_2$ as its two non-zero eigenvectors and ${\lambda }_1$ and ${\lambda }_2$ as the two distinct eigenvalues corresponding to these vectors.

If $u_1$ and $u_2$ are linearly independent, ${\alpha }_1{u}_1+{\alpha }_2{u}_2=0$ for ${\alpha }_1,{\alpha }_2\ne 0$.

$$\begin{array}{rlrl}& & {\alpha }_1{u}_1+{\alpha }_2{u}_2& =0\\ & \Rightarrow & T({\alpha }_1{u}_1+{\alpha }_2{u}_2)& =T(0)\\ & \Rightarrow & {\alpha }_{1}T(u_1)+{\alpha }_{2}T(u_2)& =0\\ & \Rightarrow & {\alpha }_1{\lambda }_1{u}_1+{\alpha }_2{\lambda }_2{u}_2& =0\end{array}\text{\hspace{1em}(1)(2)}$$

If we multiply $(1)$ by ${\lambda }_1$ and subtract the result with $(2)$, we get -

$${\alpha }_2({\lambda }_1-{\lambda }_2)u_2=0$$

Because we know that ${\lambda }_1$ and ${\lambda }_2$ are distinct and that $u_2$ is a non-zero eigenvector, ${\alpha }_2=0$. Using this information we can show that ${\alpha }_1=0$.

Thus, if the eigenvalues are distinct, the eigenvectors are linearly independent.

Diagonalization

Let $A$ be a matrix in ${\mathbb{R}}^n$ with a set of $n$ linearly independent vectors. We can make a matrix $P$ with these eigenvectors as its columns.

$$p=\left[\begin{array}{cccc}⋮& ⋮& ⋮& ⋮\\ u_1& u_2& \dots & u_n\\ ⋮& ⋮& ⋮& ⋮\end{array}\right]$$

If we do $AP$ then,

\begin{aligned} AP &= A[\begin{array}& u_1 & u_2 &\dots &u_n\end{array}] \\[8pt] &= [\begin{array} &Au_1 & Au_2 &\dots &Au_n\end{array}] \\[8pt] &= [\begin{array} &\alpha_1u_1 & \alpha_2u_2 &\dots &\alpha_nu_n\end{array}] \\[8pt] &= [\begin{array}& u_1 & u_2 &\dots &u_n\end{array}] \left[ \begin{matrix} \alpha_1 & 0 &\dots &0 \\ 0 &\alpha_2 &\dots &0 \\ \vdots&\vdots&\vdots&\vdots& \\ 0 &0 &\dots &\alpha_n \\ \end{matrix} \right] \\[8pt] &= PD \end{aligned}

So we end up with

$$\boxed{AP=PD}$$

Because $P$ is made up of $n$ linearly independent eigenvectors belonging to ${\mathbb{R}}^n$, $P$ is a square and invertible matrix. Thus we can also say,

$$\boxed{A=PD{P}^{-1}}$$

$\underline{\text{Definition}}-$ Any matrix in ${\mathbb{R}}^n$ with $n$ linearly independent eigenvectors can be decomposed as a product of matrices formed using its eigenvalues and eigenvectors. This is called the diagonalization or eigen decomposition of the matrix.

If any eigenvalue of a matrix is deficient, it’ll mean that we don’t have enough eigenvectors needed to capture the whole $n-$dimensional space. Thus in such a case the matrix is un-diagonalizable.

Powers of a diagonalizable matrix

If $A$ is diagonalizable, it can be decomposed into a product of eigen matrices. These eigen matrices can be used to write the power of matrix $A$.

$$\begin{array}{rlrl}& & A& =PD{P}^{-1}\\ & \Rightarrow & A^2& =(PD{P}^{-1})(PD{P}^{-1})\\ & & & =PD\cancel{P^{-1}P}D{P}^{-1}\\ & & & =P{D}^2{P}^{-1}\end{array}$$

This can be generalized to $\boxed{A^n=P{D}^n{P}^{-1}}$.

As $D$ is a diagonal matrix, calculating the powers of $D$ becomes trivial.

$$\begin{array}{rlrl}& & D^2& =\left[\begin{array}{ccccc}{\alpha }_1& 0& \dots & 0& \\ 0& {\alpha }_2& \dots & 0& \\ ⋮& ⋮& ⋮& ⋮& \\ 0& 0& \dots & {\alpha }_n& \end{array}\right]\left[\begin{array}{ccccc}{\alpha }_1& 0& \dots & 0& \\ 0& {\alpha }_2& \dots & 0& \\ ⋮& ⋮& ⋮& ⋮& \\ 0& 0& \dots & {\alpha }_n& \end{array}\right]\\ & & & =\left[\begin{array}{ccccc}{\alpha }_1^2& 0& \dots & 0& \\ 0& {\alpha }_2^2& \dots & 0& \\ ⋮& ⋮& ⋮& ⋮& \\ 0& 0& \dots & {\alpha }_n^2& \end{array}\right]\\ & & D^n& =\left[\begin{array}{ccccc}{\alpha }_1^n& 0& \dots & 0& \\ 0& {\alpha }_2^n& \dots & 0& \\ ⋮& ⋮& ⋮& ⋮& \\ 0& 0& \dots & {\alpha }_n^n& \end{array}\right]\end{array}$$

As $n\to \infty$,

• If any eigenvalue $|{\alpha }_i|<1$ then ${\alpha }_i^n=0$.
• If any $|{\alpha }_i|>1$ then ${\alpha }_i^n$ tends to $\pm \infty$.
• If any $|{\alpha }_i|=\pm 1$ then ${\alpha }_i^n=\pm 1$.