Matrix norms#

Frobenius norm#

Let \(\boldsymbol A \in \mathbb R^{m\times n}\). Frobenius norm of \(\boldsymbol A\) is defined as

\[ \Vert \boldsymbol A \Vert_F = \sqrt{\mathrm{tr}(\boldsymbol A^\mathsf{T} \boldsymbol A)} = \sqrt{\sum\limits_{i=1}^m \sum\limits_{j=1}^n a_{ij}^2}. \]

Spectral norm#

The spectral norm of the matrix \(\boldsymbol A \in \mathbb R^{m\times n}\) is equal to maximal eigenvalue of \(\boldsymbol A^\mathsf{T} \boldsymbol A\):

\[ \Vert \boldsymbol A \Vert_2 = \sqrt{\lambda_{\max}(\boldsymbol A^\mathsf{T} \boldsymbol A)}. \]

Condition number#

For a square matrix \(\boldsymbol A\) define its conditional number as

\[ \kappa(\boldsymbol A) = \Vert \boldsymbol A \Vert_2 \cdot\Vert\boldsymbol A^{-1}\Vert_2. \]

If \(\boldsymbol A\) is singular then \(\kappa(\boldsymbol A) = \infty\).

Properties of conditional numbers

  • \(\kappa(\boldsymbol A) \geqslant 1\);

  • \(\kappa(\boldsymbol A) = \kappa(\boldsymbol A^{-1})\);

  • \(\kappa(\boldsymbol{AB}) \leqslant \kappa(\boldsymbol A)\kappa(\boldsymbol B)\).

Exercises#

  1. Find \(\Vert \boldsymbol I_n \Vert_F\), \(\Vert \boldsymbol I_n \Vert_2\), \(\kappa(\boldsymbol I_n)\).

  2. Find \(\Vert \boldsymbol Q \Vert_F\), \(\Vert \boldsymbol Q \Vert_2\), \(\kappa(\boldsymbol Q)\) if \(\boldsymbol Q\) is an orthogonal matrix.

  3. Let \(\boldsymbol A = \boldsymbol A^\mathsf{T}\). Show that \(\Vert \boldsymbol A \Vert_2 = \max\limits_{\lambda \in \mathrm{spec}(\boldsymbol A)}\vert\lambda\vert\).

  4. Let \(\boldsymbol A\) be a symmetric positive definite matrix. Show that \(\kappa(\boldsymbol A) = \Big\vert\frac{\lambda_{\max}(\boldsymbol A)}{\lambda_{\min}(\boldsymbol A)}\Big\vert\).