User Tools

Site Tools


Math 753/853 Norms, inner products, and orthogonality

Ok, this is a big set of topics, and nothing I've found covers the topic at the right level of detail or depth. So, here's a summary of a few key points you should understand. These were spelled out in detail during lecture.

Inner product

The inner product of two vectors $x$ and $y$

x^Ty = \sum_{i=1}^m x_i y_i

where $x^T$ is the transpose of $x$. If $x^Ty = 0$, $x$ and $y$ are orthogonal.


The 2-norm of a vector $x$ is defined as

\|x\| = \sqrt{\sum_{i=1}^m x_i^2}

Note that $x^Tx = \|x\|^2$.

The 2-norm of a matrix $A$ is defined as

\|A\| = \sup_{x\neq0} \frac{\|Ax\|}{\|x\|} = \sup_{\|x\| = 1 } \|Ax\|

You can think of $\|A\|$ as the maximum amplification factor in length that can occur under the map $x \rightarrow Ax$.

Orthogonal matrices

A matrix $Q$ is an orthogonal matrix if its inverse is its transpose: $Q^T Q = I$. The columns of an orthogonal matrix are a set of orthogonal vectors.

Key properties of orthogonal matrices:

  • The inner product is preserved under orthogonal transformations: $(Qx)^T(Qy) = x^Ty$.
  • The vector 2-norm is preserved under orthogonal transformations: $\|Qx\| = \|x\|$.
  • The matrix 2-norm is preserved under orthogonal transformations: $\|QA\| = \|A\|$.
  • The 2-norm of an orthogonal matrix is one: $\|Q\| = 1$.

For further details, see the following Wikipedia pages

gibson/teaching/fall-2016/math753/norms-orthogonality.txt · Last modified: 2016/10/06 11:58 by gibson