数学建模

 

模型知识点

高等数学-线性代数-特征值与特征向量

 

Wang Haihua

🚅 🚋😜 🚑 🚔

Eigenvalue and Eigenvector

An eigenvector of an $n \times n$ matrix $A$ is a nonzero vector $x$ such that $Ax = \lambda x$ for some scalar $\lambda$. A scalar $\lambda$ is called an eigenvalue of $A$ if there is a nontrivial solution $x$ of $Ax = \lambda x$, such an $x$ is called an eigenvector corresponding to $\lambda$.

Rewrite the equation,

$$ (A-\lambda I)x = 0 $$

Since the eigenvector should be a nonzero vector, which means:

1. The column or rows of $(A-\lambda I)$ are linearly dependent
2. $(A-\lambda I)$ is not full rank, $Rank(A)<n$.
3. $(A-\lambda I)$ is not invertible.
4. $\text{det}(A-\lambda I)=0$, which is called characteristic equation.

Consider a matrix $A$

$$ A=\left[\matrix{1 & 0 & 0\cr 1 & 0 & 1\cr 2 & -2 & 3}\right] $$

Set up the characteristic equation,

$$ \text{det}\left( \left[\matrix{1 & 0 & 0\cr 1 & 0 & 1\cr 2 & -2 & 3}\right]- \lambda \left[\matrix{1 & 0 & 0\cr 0 & 1 & 0\cr 0 & 0 & 1}\right] \right)=0 $$

To find the eigenvector corresponding to $\lambda$, we substitute the eigenvalues back into $(A-\lambda I)x=0$ and solve it. Construct augmented matrix with $\lambda =1$ and perform rref.

The null space is the solution set of the linear system.

$$ \left[ \begin{matrix} x_1 \\ x_2 \\ x_3 \end{matrix} \right]= \left[ \begin{matrix} x_2-x_3 \\ x_2 \\ x_3 \end{matrix} \right]= x_2\left[ \begin{matrix} 1 \\ 1 \\ 0 \end{matrix} \right] +x_3\left[ \begin{matrix} -1 \\ 0 \\ 1 \end{matrix} \right] $$

This is called eigenspace for $\lambda = 1$, which is a subspace in $\mathbb{R}^3$. All eigenvectors are inside the eigenspace.

We can proceed with $\lambda = 2$ as well.

The null space is the solution set of the linear system.

$$ \left[ \begin{matrix} x_1 \\ x_2 \\ x_3 \end{matrix} \right]= \left[ \begin{matrix} 0\\ \frac{1}{2}x_3\\ x_3 \end{matrix} \right]= x_3\left[ \begin{matrix} 0 \\ \frac{1}{2} \\ 1 \end{matrix} \right] $$

An Example

Consider a matrix $A$

A = np.array(A)
A = A.astype(float)
eigval, eigvec = np.linalg.eig(A)
eigval

array([ 21.69, -16.69,   3.  ,   2.  ,   2.  ])

A Visualization Example

Let

$$ A= \left[ \begin{matrix} 1 & 6\\ 5 & 2 \end{matrix} \right] $$

find the eigenvalues and vectors, then visualize in $\mathbb{R}^2$

Use characteristic equation $|A - \lambda I|=0$

$$ \left| \left[ \begin{matrix} 1 & 6\\ 5 & 2 \end{matrix} \right] - \left[ \begin{matrix} \lambda & 0\\ 0 & \lambda \end{matrix} \right]\right|=0 $$

lamda = sy.symbols('lamda')
A = sy.Matrix([[1,6],[5,2]])
I = sy.eye(2)

A - lamda*I

$\displaystyle \left[\begin{matrix}1 - \lambda & 6\\5 & 2 - \lambda\end{matrix}\right]$

p = A.charpoly(lamda);p

$\displaystyle \operatorname{PurePoly}{\left( \lambda^{2} - 3 \lambda - 28, \lambda, domain=\mathbb{Z} \right)}$

sy.factor(p)

$\displaystyle \left(\lambda - 7\right) \left(\lambda + 4\right)$

There are two eigenvalues: $7$ and $4$. Next we calculate eigenvectors.

(A - 7*sy.eye(2)).row_join(sy.zeros(2,1)).rref()

$\displaystyle \left( \left[\begin{matrix}1 & -1 & 0\\0 & 0 & 0\end{matrix}\right], \ \left( 0\right)\right)$

The eigenspace for $\lambda = 7$ is

$$ \left[ \begin{matrix} x_1\\ x_2 \end{matrix} \right]= x_2\left[ \begin{matrix} 1\\ 1 \end{matrix} \right] $$

Any vector is eigenspace as long as $x \neq 0$ is an eigenvector. Let's find out eigenspace for $\lambda = 4$.

The eigenspace for $\lambda = -4$ is

$$ \left[ \begin{matrix} x_1\\ x_2 \end{matrix} \right]= x_2\left[ \begin{matrix} -\frac{6}{5}\\ 1 \end{matrix} \right] $$

Let's plot both eigenvectors as $(1, 1)$ and $(-6/5, 1)$ and multiples with eigenvalues.

ax.minorticks_on()
ax.tick_params(axis = 'both', direction = 'inout', length=12, width=2, which='major')
ax.tick_params(axis = 'both', direction = 'inout', length=10, width=1, which='minor')
ax.grid()
plt.show()

Geometric Intuition

Eigenvector has a special property that preserves the pointing direction after linear transformation.To illustrate the idea, let's plot a 'circle' and arrows touching edges of circle.

Start from one arrow. If you want to draw a smoother circle, you can use parametric function rather two quadratic functions, because cicle can't be draw with one-to-one mapping.But this is not the main issue, we will live with that.

x = np.linspace(-4, 4)
y_u = np.sqrt(16 - x**2)
y_d = -np.sqrt(16 - x**2)

fig, ax = plt.subplots(figsize = (8, 8))
ax.plot(x, y_u, color = 'b')
ax.plot(x, y_d, color = 'b')

ax.scatter(0, 0, s = 100, fc = 'k', ec = 'r')

ax.arrow(0, 0, x[5], y_u[5], head_width = .18, 
         head_length= .27, length_includes_head = True, 
         width = .03, ec = 'r', fc = 'None')
plt.show()

Now, the same 'circle', but more arrows.

x = np.linspace(-4, 4, 50)
y_u = np.sqrt(16 - x**2)
y_d = -np.sqrt(16 - x**2)

fig, ax = plt.subplots(figsize = (8, 8))
ax.plot(x, y_u, color = 'b')
ax.plot(x, y_d, color = 'b')

ax.scatter(0, 0, s = 100, fc = 'k', ec = 'r')

for i in range(len(x)):
    ax.arrow(0, 0, x[i], y_u[i], head_width = .08, 
             head_length= .27, length_includes_head = True,
             width = .01, ec = 'r', fc = 'None')
    ax.arrow(0, 0, x[i], y_d[i], head_width = .08, 
             head_length= .27, length_includes_head = True, 
             width = .008, ec = 'r', fc = 'None')

Now we will perform linear transformation on the circle. Technically, we can only transform the points - the arrow tip - that we specify on the circle.

We create a matrix

$$ A = \left[\begin{matrix} 3 & -2\\ 1 & 0 \end{matrix}\right] $$

Align all the coordinates into two matrices for upper and lower half respectively.

$$ V_u = \left[\begin{matrix} x_1^u & x_2^u & \ldots & x_m^u\\ y_1^u & y_2^u & \ldots & y_m^u \end{matrix}\right]\\ V_d = \left[\begin{matrix} x_1^d & x_2^d & \ldots & x_m^d\\ y_1^d & y_2^d & \ldots & y_m^d \end{matrix}\right] $$

The matrix multiplication $AV_u$ and $AV_d$ are linear transformation of the circle.

The circle becomes an ellipse. However, if you watch closely, you will find there are some arrows still pointing the same direction after linear transformation.

plt.show()

We can plot the cirle and ellipse together, those vectors pointing the same direction before and after the linear transformation are eigenvector of $A$, eigenvalue is the length ratio between them.

plt.show()