Symmetric Matrices

Say we have a

n\times n

matrix

\(A\)

and

a_{ij}\in\mathbb{R}

\(A\)

is symmetric if

\(A=A^T\)

A=\begin{bmatrix} a_{11} & \color{red}{a_{12}} & \cdots & \color{blue}{a_{1n}} \\ \color{red}{a_{21}} & a_{22} & \cdots & \color{darkgreen}{a_{2n}} \\ \vdots & \vdots & \ddots & \vdots \\ \color{blue}{a_{n1}} & \color{darkgreen}{a_{n2}} & \cdots & a_{nn} \\ \end{bmatrix} \\ \quad \\ a_{ij}=a_{ji}

Properties of Symmetric matrices

$1.\quad$ Eigenvalues of symmetric matrices are real

Explanation:

Say $\lambda\in\mathbb{C}$ is the eigenvalues and $x\in\mathbb{C}^n$ is the eigenvector of matrix $$A$$ so,
(Here $\mathbb{C}$ represent Complex numbers)
$Ax=\lambda x$
$\Rightarrow$ $\left(\overline{x}\right)^TAx=\lambda\ (\overline{x})^Tx$ $\color{red}{(1)}$

$$($$ And this bar( $\overline{ \ \ }$ ) means that we are taking conjugate of complex number, like, $\overline{a+ib}=a-ib)$

$Ax=\lambda x$
$\Rightarrow \overline{A}\ \overline{x}=\overline{\lambda}\ \overline{x}$
because element of $$A$$ are Real numbers so $\overline{A}=A$
$\Rightarrow A\ \overline{x}=\overline{\lambda}\ \overline{x}$
$\Rightarrow (A\ \overline{x})^T=(\overline{\lambda}\ \overline{x})^T$
$\displaystyle \Rightarrow \left(\overline{x}\right)^TA^T=\overline{\lambda}\ (\overline{x})^T$
because $$A$$ is a symmetric matrix so,
$\displaystyle \Rightarrow \left(\overline{x}\right)^TA=\overline{\lambda}\ (\overline{x})^T$
$\Rightarrow$ $\left(\overline{x}\right)^TAx=\overline{\lambda}\ (\overline{x})^Tx$ $\color{red}{(2)}$
Using above equations $\color{red}{(1)}$ and $\color{red}{(2)}$ we can say that,
$\lambda\ (\overline{x})^Tx = \overline{\lambda}\ (\overline{x})^Tx$
$\Rightarrow$ $\lambda=\overline{\lambda}$
It is only possible if complex part of $\lambda$ is $$0$$ so eigenvalues are indeed Real. ✅

$2.\quad$ For a symmetric matrix we can get orthonormal eigenvectors

Explanation:

Say $\vec{q}_1, \vec{q}_2, \cdots, \vec{q}_n$ are eigenvectors of $$A$$
Say, $Q=\begin{bmatrix} \vdots & \vdots & & \vdots \\ q_1 & q_2 & \cdots & q_n \\ \vdots & \vdots & & \vdots \\ \end{bmatrix}$ So $A=Q\Lambda Q^{-1}$ we discussed it HERE
And because $$Q$$ is an orthonormal matrix so $Q^{-1}=Q^T$ we discussed it HERE so,
$\begin{matrix} A=Q\Lambda Q^{T} \end{matrix}$
So we can write it as,
$A= \begin{bmatrix} \vdots & \vdots & & \vdots \\ \vec{q}_{1} & \vec{q}_{2} & \cdots & \vec{q}_{n} \\ \vdots & \vdots & & \vdots \\ \end{bmatrix} {\begin{bmatrix} \lambda_1 & 0 & \cdots & 0 \\ 0 & \lambda_2 & \cdots & 0 \\ \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & \cdots & \lambda_n \\ \end{bmatrix}} \begin{bmatrix} \cdots q_1 \cdots\\ \cdots q_2 \cdots\\ \vdots\\ \cdots q_n \cdots\\ \end{bmatrix}$
So,
$\begin{matrix} \displaystyle A=\lambda_1q_1q_1^T + \lambda_2q_2q_2^T + \cdots + \lambda_nq_nq_n^T \end{matrix}$
Here all $$q_i$$ 's are orthonormal vectors so $$q_iq_i^T$$ is a projection matrix so we can say that,
$$A$$ is the combination of orthogonal projection matrices.
(we talked about Orthogonal Subspaces HERE)

$3.\quad$ For a symmetric matrix signs of pivots are same as signs of eigenvalues

Explanation:

Say that we have a $50\times 50$ symmetric matrix and we just want to know there signs.
In differential equation we saw that knowing just the signs of eigenvalues are important, it tells us about the state of the system.
Here we can't go for $\text{det}(A-\lambda\mathcal{I})=0$ , because it will give us a $$50$$ degree polynomial, and we can spend lifetime solving this.
So what we do instead is we find $$50$$ pivots and find number of positive pivots and number of negative pivots.
And for a Symmetric matrix,
$\mathbf{\#}$ of positive pivots $$=$$ $\mathbf{\#}$ of positive eigenvalues
$\mathbf{\#}$ of negative pivots $$=$$ $\mathbf{\#}$ of negative eigenvalues

Positive Definite Symmetric Matrices

Facts:
For a Symmetric Matrix to be Positive Definite all eigenvalues must be positive.
If all eigenvalues are positive then all pivots are also positive.
For a Symmetric Matrix to be Positive Definite all leading determinant must be positive.

Symmetric Matrices

Properties of Symmetric matrices

\[ \begin{matrix} A=Q\Lambda Q^{T} \end{matrix} \]

Positive Definite Symmetric Matrices

$\begin{matrix} A=Q\Lambda Q^{T} \end{matrix}$