Subsection
This section is contributed by \elizabethmillion.
You may have once thought that the natural definition for matrix multiplication would be entrywise multiplication, much in the same way that a young child might say, “I writed my name.” The mistake is understandable, but it still makes us cringe. Unlike poor grammar, however, entrywise matrix multiplication has reason to be studied; it has nice properties in matrix analysis and additionally plays a role with relative gain arrays in chemical engineering, covariance matrices in probability and serves as an inertia preserver for Hermitian matrices in physics. Here we will only explore the properties of the Hadamard product in matrix analysis.
Definition HP Hadamard Product
Let $A$ and $B$ be $m \times n$ matrices. The Hadamard Product of $A$ and $B$ is defined by $\matrixentry{\hadamard{A}{B}}{ij} = \matrixentry{A}{ij} \matrixentry{B}{ij}$ for all $1 \leq i \leq m$, $1 \leq j \leq n$.
$\hadamard{A}{B}$
$\square$
As we can see, the Hadamard product is simply “entrywise multiplication”. Because of this, the Hadamard product inherits the same benefits (and restrictions) of multiplication in \complexes. Note also that both $A$ and $B$ need to be the same size, but not necessarily square. To avoid confusion, juxtaposition of matrices will imply the “usual” matrix multiplication, and we will use “$\circ$” for the Hadamard product.
Now we will explore some basics properties of the Hadamard Product.
Theorem HPC Hadamard Product is Commutative
If $A$ and $B$ are $m \times n$ matrices then $\hadamard{A}{B} = \hadamard{B}{A}$.
Proof
Definition HID Hadamard Identity
The Hadamard identity is the $m \times n$ matrix \hadamardidentity{mn} defined by $\matrixentry{\hadamardidentity{mn}}{ij} = 1$ for all $1 \leq i \leq m$, $1 \leq j \leq n$.
$\hadamardidentity{mn}$
$\square$
Theorem HPHID Hadamard Product with the Hadamard Identity
Suppose $A$ is an $m \times n$ matrix. Then $\hadamard{A}{\hadamardidentity{mn}} = \hadamard{\hadamardidentity{mn}}{A} = A$.
Proof
Definition HI Hadamard Inverse
Let $A$ be an $m \times n$ matrix and suppose $\matrixentry{A}{ij} \not= 0$ for all $1 \leq i \leq m$, $1 \leq j \leq n$. Then the Hadamard Inverse, \hadamardinverse{A} , is given by $\matrixentry{\hadamardinverse{A}}{ij} = (\matrixentry{A}{ij})^{-1}$ for all $1 \leq i \leq m$, $1 \leq j \leq n$.
$\hadamardinverse{A}$
$\square$
Theorem HPHI Hadamard Product with Hadamard Inverses
Let $A$ be an $m \times n$ matrix such that $\matrixentry{A}{ij} \not= 0$ for all $1 \leq i \leq m$, $1 \leq j \leq n$. Then $\hadamard{A}{\hadamardinverse{A}} = \hadamard{\hadamardinverse{A}}{A} = \hadamardidentity{mn}$.
Proof
Since matrices have a different inverse and identity under the Hadamard product, we have used special notation to distinguish them from what we have been using with “normal” matrix multiplication. That is, compare “usual” matrix inverse, \inverse{A}, with the Hadamard inverse \hadamardinverse{A}, and the “usual” matrix identity, $I_{n}$, with the Hadamard identity, \hadamardidentity{mn}.
The Hadamard identity matrix and the Hadamard inverse are both more limiting than helpful, so we will not explore their use further. One last fun fact for those of you who may be familiar with group theory: the set of $m \times n$ matrices with nonzero entries form an abelian (commutative) group under the Hadamard product (prove this!).
Theorem HPDAA Hadamard Product Distributes Across Addition
Suppose $A$, $B$ and $C$ are $m \times n$ matrices. Then $\hadamard{C}{(A + B)} = \hadamard{C}{A} + \hadamard{C}{B}$.
Proof
Theorem HPSMM Hadamard Product and Scalar Matrix Multiplication
Suppose $\alpha \in \complexes$, and $A$ and $B$ are $m \times n$ matrices. Then $\alpha (\hadamard{A}{B}) = \hadamard{(\alpha A)}{B} = \hadamard{A}{(\alpha B)}$.
Proof
\subsect{DMHP}{Diagonal Matrices and the Hadamard Product}
We can relate the Hadamard product with matrix multiplication by considering diagonal matrices, since $A \circ B = AB$ if and only if both $A$ and $B$ are diagonal. (Can you prove this?) For example, a simple calculation reveals that the Hadamard product relates the diagonal values of a diagonalizable matrix $A$ with its eigenvalues:
Theorem DMHP Diagonalizable Matrices and the Hadamard Product
Let $A$ be a diagonalizable matrix of size $n$ with eigenvalues \scalarlist{\lambda}{n}. Let $D$ be a diagonal matrix from the diagonalization of $A$, $A = SDS^{-1}$, and \vect{d} be a vector such that \matrixentry{D}{ii} $=$\vectorentry{\vect{d}}{i}$= \lambda_i$ for all $1 \leq i \leq n$. Then
\begin{align*}
\matrixentry{A}{ii}
&=
\vectorentry{\hadamard{S}{\transpose{(\inverse{S})}} \vect{d}}{i}
& \text{for all }1 \leq i \leq n.
\end{align*}
That is,
\begin{equation*}
\colvector{\matrixentry{A}{11}\\ \matrixentry{A}{22}\\ \matrixentry{A}{33}\\ \vdots \\ \matrixentry{A}{nn}} % Is this an OK way to handle this vector?
=
\hadamard{S}{\transpose{(\inverse{S})}} \vectorcomponents{\lambda}{n}
\end{equation*}
Proof
We obtain a similar result when we look at the singular value decomposition of square matrices (see exercises).
Theorem DMMP Diagonal Matrices and Matrix Products
Suppose $A$, $B$ are $m \times n$ matrices, and $D$ and $E$ are diagonal matrices of size $m$ and $n$, respectively. Then,
\begin{equation*}
D(\hadamard{A}{B})E
=
\hadamard{(DAE)}{B}
=
\hadamard{(DA)}{(BE)}
\end{equation*}
Proof
