Rank of matrices

Consider a

m\times n

matrix

\(A\)

of rank

\(r\)

.
So we have

\(n\)

column vectors in

\mathbb{R}^m

, and there are

\((n-r)\)

column vectors which we can get by the linear combination of other

\(r\)

column vectors.
We describe rank as number of pivots of a matrix.

# pivots can't be $\gt$ # rows ( $$m$$ ), so $r\leq m$
# pivots can't be $\gt$ # columns ( $$n$$ ), so $r\leq n$

▶ Full column rank matrix

Consider a

m\times n

matrix

\(A\)

of rank

\(r\)

and

m\gt n

.
Here

\text{Rank}(A)=r=n

it means that all of the column vectors are independent, # pivots

\(=n\)

\Rightarrow

# free column vectors

\(=0\)

Can we always get a solution, can we always find $$x$$ such that $A\vec{x}=\vec{b}$ for every $\vec{b}$ ?
Here in $$A$$ we have $$n$$ independent vectors in $\mathbb{R}^m$ and $m\gt n$ , to fill a $$m$$ dimensional space we alteast need $$m$$ independent vectors, but we had $$n$$ independent vectors and $n\lt m$ so we don't have enough vectors to fill $\mathbb{R}^m$ .
Linear combination of $$n$$ independent vectors can't give us a space in $\mathbb{R}^m$
So we can't get every vector $\vec{b}$ by the linear combinations of those independent vectors.
So answer is No we can't get every $\vec{b}$ .

We have no free vectors here so Null Space of

\(A\)

is just

\vec{0}

, so

A\vec{x}=\vec{0}

has only one solution that is

\vec{x}=\vec{0}

Complete solution of

A\vec{x}=\vec{b}

\vec{x}=\vec{x}_p+\vec{x}_n=\vec{x}_p+\vec{0}=\vec{x}_p

So complete solution is just a single vector $\vec{x}_p$ if it exists.
# solution = $\left\{\begin{matrix} 1 & \text{ if there is a solution} \\ 0 & \text{ otherwise} \\ \end{matrix}\right.$

Example:

A = \begin{bmatrix} 1 & 4\\ 2 & 3\\ 3 & 2\\ 4 & 1\\ \end{bmatrix}

Can we always get a solution, can we always find $$x$$ such that $A\vec{x}=\vec{b}$ for every $\vec{b}$ ?
Here in $$A$$ we have $$2$$ vectors in $\mathbb{R}^4$ .
and linear combination of $$2$$ vector can give us a space in $\mathbb{R}^2$ so we can only get those $\vec{b}$ which are in the linear combination(plane) of those two vectors.
possible $\vec{b}$ is $\vec{b}= \alpha\begin{bmatrix} 1\\2\\3\\4\\ \end{bmatrix} +\beta\begin{bmatrix} 4\\3\\2\\1\\ \end{bmatrix};\quad$ $\alpha,\beta\in\mathbb{R}$
So answer is No we can't get every $\vec{b}$

row reduced echelon form is

\begin{bmatrix} \fbox{1} & 0 \\ 0 & \fbox{1} \\ 0 & 0 \\ 0 & 0 \\ \end{bmatrix} = \begin{bmatrix} I_{2\times2}\\0 \end{bmatrix}

So # of pivots are

\(2\)

▶ Full row rank matrix

Consider a

m\times n

matrix

\(A\)

of rank

\(r\)

and

m\lt n

.
Here

\text{Rank}(A)=r=m

now all of the column vectors are not independent, # pivots

\(=m\)

\Rightarrow

# free column vectors

\(=n-r\)

Can we always get a solution, can we always find $$x$$ such that $A\vec{x}=\vec{b}$ for every $\vec{b}$ ?
Here in $$A$$ we have $$m$$ independent vectors in $\mathbb{R}^m$ , so we have enough vectors to fill $\mathbb{R}^m$ .
Linear combination of $$m$$ independent vectors can give us a space in $\mathbb{R}^m$
So we can get every vector $\vec{b}$ by the linear combinations of those independent vectors.
So answer is Yes we can get every $\vec{b}$ .

Here we have

\(n-r\)

free vectors so Null Space of

\(A\)

is in

\mathbb{R}^{n-r}

# solutions are $\infty$

Example:

A = \begin{bmatrix} 1 & 2 & 3 & 4 \\ 4 & 3 & 2 & 1 \\ \end{bmatrix}

Can we always get a solution, can we always find $$x$$ such that $A\vec{x}=\vec{b}$ for every $\vec{b}$ ?
Here in $$A$$ we have $$4$$ vectors in $\mathbb{R}^2$ and out of those $$2$$ vectors are independent.
And linear combination of $$2$$ independent vector can give us a space in $\mathbb{R}^2$ . So we covered the whole $$2$$ -dimensional space.
So answer is Yes we can get every $\vec{b}$ .

row reduced echelon form is

\begin{bmatrix} \fbox{1} & 0 & -1 & -2 \\ 0 & \fbox{1} & 2 & 3 \\ \end{bmatrix} = \begin{bmatrix} I_{2\times2} & F_{2\times2} \end{bmatrix}

So # of pivots are

\(2\)

▶ Full rank matrix

Consider a

m\times n

matrix

\(A\)

of rank

\(r\)

and

\(m = n\)

.
Here

\text{Rank}(A)=r=n

it means that all of the column vectors are independent, # pivots

\(=n\)

\Rightarrow

# free column vectors

\(=0\)

Can we always get a solution, can we always find $$x$$ such that $A\vec{x}=\vec{b}$ for every $\vec{b}$ ?
Here in $$A$$ we have $$n$$ independent vectors in $\mathbb{R}^n$ , so we have enough vectors to fill $\mathbb{R}^n$ .
Linear combination of $$n$$ independent vectors can give us a space in $\mathbb{R}^n$
So we can get every vector $\vec{b}$ by the linear combinations of those independent vectors.
So answer is Yes we can get every $\vec{b}$ .

We have no free vectors here so Null Space of

\(A\)

is just

\vec{0}

, so

A\vec{x}=\vec{0}

has only one solution that is

\vec{x}=\vec{0}

# solutions = $$1$$
row reduced echelon form of $$A$$ is $I_{n\times n}$

Dimensions and Basis

Say we have a matrix

A_{m\times n}\in\mathbb{R}^{m\times n}

▶ Column Space

We know that the column space of matrix

\(A\)

lives in

R^{m}

.
But what are the basis for that column space?

Basis are those (minimum) vectors whose linear combination gives us the desired space.
for example, for space of $$R^2$$ , linear combinations of any $$2$$ non-parallel(independent) vectors will give us $$R^2$$ .

The basis for column space are those (independent)pivot columns

What is the Dimension of column space?

Number of dimensions are defined by it's basis vectors so, Number of dimensions $$=$$ number of pivot columns $$=$$ rank $$(r)$$ so,
# Dimensions $$= r$$

▶ Null Space

Null space of matrix

\(A\)

also lives in

R^{m}

Recall those free variables of matrix $$A$$ , free variables are the dependent column vectors of matrix $$A$$ , these free variables gives us the special solutions and linear combination of those special solutions gives us our Null space.

What are the basis for that Null space?

We discussed that Null space is the linear combination of those special solutions so,
The basis for Null space are those special solutions.

What is the Dimension of Null space?

Number of dimensions are defined by it's basis vectors so, Number of dimensions $$=$$ number of special solution and we discussed in Null Space section that, Number of special solution are $$n-r$$ .
# Dimensions $$= n-r$$