Eigen Decomposition

序言：對於對稱矩陣 A，可以運用 Eigen Decomposition 分解成 $UDU^T$

1. For A with shape (n, n): exist n complex eigenvalues and eigenvectors such that $A\vec{u} = \lambda \vec{u}$

$A\vec{u} = \lambda \vec{u} \rightarrow (A-\lambda I) \vec{u} = \vec{0} \rightarrow A-\lambda I$ not invertible
$\rightarrow det(A-\lambda I)=0$  (一元n次聯立式)
$\rightarrow \lambda$有n根

2. For symmetric A, given vectors corresponding to distinct eigenvalues are orthogonal.

sol. $A\vec{u_1} = \lambda_1 \vec{u_1}$ and $A\vec{u_2}= \lambda_2 \vec{u_2}, \lambda_1 \neq \lambda_2$

$\lambda_1 \vec{u_1}^T \vec{u_2} = (\lambda_1 \vec{u_1})^T\vec{u_2} = (A\vec{u_1})^T\vec{u_2} = \vec{u_1}^TA\vec{u_2} = \vec{u_1}^T\lambda_{2} \vec{u_2} = \lambda_{2} \vec{u_1}^T \vec{u_2} \rightarrow \vec{u_1}^T \vec{u_2}=0$

3. For symmetric A (and A is real), eigenvalues are real.

sol. $Ax=\lambda x \rightarrow A\overline{x}=\lambda \overline{x}$ 

$(since \overline{(a+bi)(c+di)} = \overline{(ac-bd)+(ad+bc)i} = (a-bi)(c-di))$

$\overline{x^T}_{1\times n}A_{n \times n}x_{n \times 1} - x^{T}A \overline{x} = (\lambda-\overline{\lambda})\overline{x^T}x \rightarrow \lambda = \overline{\lambda}$ and $\lambda$ is real.

For symmetric A, 
$A\vec{u_1} = \lambda_1\vec{u_1}; A\vec{u_2} = \lambda_2\vec{u_2}$... (all $\vec{u_i}$ can be set to have length =1 an advance)
$\rightarrow AU=UD; A = UDU^{-1}$
又：$U^{T}U = I$ since 2. $\rightarrow U^{T}= U^{-1} \rightarrow A=UDU^T$

which satisties:
1. columns in U are orthogonal to each other
2.  values in D are real