7.3: Properties of Matrices

David Cherney, Tom Denton, & Andrew Waldron
University of California, Davis

The objects of study in linear algebra are linear operators. We have seen that linear operators can be represented as matrices through choices of ordered bases, and that matrices provide a means of efficient computation. We now begin an in depth study of matrices.

Definition: matrix, Column and Row Vectors An $r \times k$ matrix $M=(m^_)$ for $i=1, \ldots, r; j=1, \ldots, k$ is a rectangular array of real (or complex) numbers: \[M =
\begin
m_^ & m_^ & \cdots & m_^ \\
m_^ & m_^ & \cdots & m_^ \\
\vdots & \vdots & & \vdots \\
m_^ & m_^ & \cdots & m_^ \\
\end\, .
\] The numbers $m^_$ are called entries. The superscript indexes the row of the matrix and the subscript indexes the column of the matrix in which $m_^$ appears. An $r\times 1$ matrix $v = (v^_) = (v^)$ is called a column vector, written \[v = \beginv^\\v^\\ \vdots \\ v^ \end\, .\] A $1\times k$ matrix $v = (v^_) = (v_)$ is called a row vector, written \[v = \beginv_ & v_ & \cdots & v_ \end\, .\]

The transpose of a column vector is the corresponding row vector and vice versa: A matrix is an efficient way to store information:
Example $\PageIndex<2>$: Gif images In computer graphics, you may have encountered image files with a .gif extension. These files are actually just matrices: at the start of the file the size of the matrix is given, after which each number is a matrix entry indicating the color of a particular pixel in the image. This matrix then has its rows shuffled a bit: by listing, say, every eighth row, a web browser downloading the file can start displaying an incomplete version of the picture before the download is complete. Finally, a compression algorithm is applied to the matrix to reduce the file size.

$151166977515379.png$
Example $\PageIndex<3>$: Graphs occur in many applications, ranging from telephone networks to airline routes. In the subject of graph theory, a graph is just a collection of vertices and some edges connecting vertices. A matrix can be used to indicate how many edges attach one vertex to another. For example, the graph pictured above would have the following matrix, where $m^_$ indicates the number of edges between the vertices labeled $i$ and $j$: \[
M = \begin
1 & 2 & 1 & 1 \\
2 & 0 & 1 & 0 \\
1 & 1 & 0 & 1 \\
1 & 0 & 1 & 3 \\
\end
\] This is an example of a $\textit$, since $m_^ = m_^$. The set of all $r\times k$ matrices \[\mathbb_^:=\<(m^_)|m^_\in \mathbb;\, i=1,\ldots,r;\, j=1\ldots k\>\, ,\] is itself a vector space with addition and scalar multiplication defined as follows: \[M+N = (m_^) + (n_^) = ( m_^ + n_^ )\] \[rM = r(m_^) = (rm_^)\] In other words, addition just adds corresponding entries in two matrices, and scalar multiplication multiplies every entry.
Notice that $M_^ = \Re^$ is just the vector space of column vectors. Recall that we can multiply an $r \times k$ matrix by a $k \times 1$ column vector to produce a $r \times 1$ column vector using the rule \[MV = \left(\sum_^ m_^ v^\right)\, .\] This suggests the rule for multiplying an $r \times k$ matrix $M$ by a $k \times s$ matrix~$N$: our $k \times s$ matrix $N$ consists of $s$ column vectors side-by-side, each of dimension $k \times 1.$ We can multiply our $r \times k$ matrix $M$ by each of these $s$ column vectors using the rule we already know, obtaining $s$ column vectors each of dimension $r \times 1.$ If we place these $s$ column vectors side-by-side, we obtain an $r \times s$ matrix $MN.$ That is, let \[N =
\begin
n_^ & n_^ & \cdots & n_^ \\
n_^ & n_^ & \cdots & n_^ \\
\vdots & \vdots & & \vdots \\
n_^ & n_^ & \cdots & n_^ \\
\end
\] and call the columns $N_$ through $N_$: \[N_ = \beginn_^\\n_^\\\vdots\\n_^\end\, ,\:
N_ = \beginn_^\\n_^\\\vdots\\n_^\end\, ,\:
\ldots,\:
N_ = \beginn_^\\n_^\\\vdots\\n_^\end.
\] Then \[
MN=M
\begin
| & | & & | \\
N_ & N_ & \cdots & N_ \\
| & | & & | \\
\end
=
\begin
| & | & & | \\
MN_ & MN_ & \cdots & MN_ \\
| & | & & | \\
\end
\] Concisely: If $M=(m^_)$ for $i=1, \ldots, r; j=1, \ldots, k$ and $N=(n^_)$ for $i=1, \ldots, k; j=1, \ldots, s,$ then $MN=L$ where $L=(\ell^_)$ for $i=i, \ldots, r; j=1, \ldots, s$ is given by \[\ell^_ = \sum_^ m^_
n^
_.\] This rule obeys linearity.

Notice that in order for the multiplication make sense, the columns and rows must match. For an $r\times k$ matrix $M$ and an $s\times m$ matrix $N$, then to make the product $MN$ we must have $k=s$. Likewise, for the product $NM$, it is required that $m=r$. A common shorthand for keeping track of the sizes of the matrices involved in a given product is: \[\left(r \times k\right)\times \left(k\times m\right) = \left(r\times m\right)\]

Example $\PageIndex<4>$: Multiplying a $(3\times 1)$ matrix and a $(1\times 2)$ matrix yields a $(3\times 2)$ matrix. \[
\begin1\\3\\2\end \begin2 & 3\end =
\begin
1\cdot 2 & 1\cdot 3 \\
3\cdot 2 & 3\cdot 3 \\
2\cdot 2 & 2\cdot 3 \\
\end
= \begin
2 & 3 \\
6 & 9 \\
4 & 6 \\
\end
\]
Another way to view matrix multiplication is in terms of dot products: \[\textitthe rows of $M$ with the columns of $N$.>\] This fact has an obvious yet important consequence:
Theorem: orthogonal Let $M$ be a matrix and $x$ a column vector. If \[
Mx=0
\] then the vector $x$ is orthogonal to the rows of $M$.

Remark Remember that the set of all vectors that can be obtained by adding up scalar multiples of the columns of a matrix is called its $\textit$. Similarly the $\textit$ is the set of all row vectors obtained by adding up multiples of the rows of a matrix. The above theorem says that if $Mx=0$, then the vector $x$ is orthogonal to every vector in the row space of $M$.

We know that $r\times k$ matrices can be used to represent linear transformations $\Re^ \rightarrow \Re^$ via $$MV = \sum_^ m_^v^ , $$ which is the same rule used when we multiply an $r\times k$ matrix by a $k\times 1$ vector to produce an $r\times1$ vector. Likewise, we can use a matrix $N=(n^_)$ to define a linear transformation of a vector space of matrices. For example
\[
L \colon M^_ \stackrel <\longrightarrow>M^_\, ,
\]
\[
L(M)=(l^_) \mbox < where >l^_= \sum_^ n_^m^_.
\]
This is the same as the rule we use to multiply matrices. In other words, $L(M)=NM$ is a linear transformation.

Matrix Terminology

Let $M=(m^_)$ be a matrix. The entries $m_^$ are called $\textit$, and the set $\^$, $m_^$, $\ldots \>$ is called the $\textit$. Any $r\times r$ matrix is called a $\textit$. A square matrix that is zero for all non-diagonal entries is called a diagonal matrix. An example of a square diagonal matrix is
$$\begin
2 & 0 & 0\\
0 & 3 & 0\\
0 & 0 & 0\\
\end\, .\] The $r\times r$ diagonal matrix with all diagonal entries equal to $1$ is called the $\textit$, $I_$, or just $I$. An identity matrix looks like \[ I=
\begin
1 & 0 & 0 & \cdots & 0 \\
0 & 1 & 0 & \cdots & 0 \\
0 & 0 & 1 & \cdots & 0 \\
\vdots & \vdots & \vdots & \ddots & \vdots \\
0 & 0 & 0 & \cdots & 1
\end.
\] The identity matrix is special because $$I_M=MI_=M$$ for all $M$ of size $r\times k$.

Definition The $\textit$ of an $r\times k$ matrix $M = (m_^)$ is the $k\times r$ matrix with entries
\[
M^ = (\hat_^)
\]
with $\hat_^ = m_^$.
A matrix $M$ is $\textit$ if $M=M^$.
Observations

Only square matrices can be symmetric.

The transpose of a column vector is a row vector, and vice-versa.

Taking the transpose of a matrix twice does nothing. \emph $(M^T)^T=M$.

Theorem: Transpose and Multiplication

Let $M, N$ be matrices such that $MN$ makes sense. Then