Math 753/853 Norms, inner products, and orthogonality

Ok, this is a big set of topics, and nothing I've found covers the topic at the right level of detail or depth. So, here's a summary of a few key points you should understand. These were spelled out in detail during lecture.

Inner product

The inner product of two vectors $x$ and $y$

$\begin{equation*} x^Ty = \sum_{i=1}^m x_i y_i \end{equation*}$

where $x^T$ is the transpose of $x$ . If $x^Ty = 0$ , $x$ and $y$ are orthogonal.

2-norm

The 2-norm of a vector $x$ is defined as

$\begin{equation*} \|x\| = \sqrt{\sum_{i=1}^m x_i^2} \end{equation*}$

Note that $x^Tx = \|x\|^2$ .

The 2-norm of a matrix $A$ is defined as

$\begin{equation*} \|A\| = \sup_{x\neq0} \frac{\|Ax\|}{\|x\|} = \sup_{\|x\| = 1 } \|Ax\| \end{equation*}$

You can think of $\|A\|$ as the maximum amplification factor in length that can occur under the map $x \rightarrow Ax$ .

Orthogonal matrices

A matrix $Q$ is an orthogonal matrix if its inverse is its transpose: $Q^T Q = I$ . The columns of an orthogonal matrix are a set of orthogonal vectors.

Key properties of orthogonal matrices:

The inner product is preserved under orthogonal transformations: $(Qx)^T(Qy) = x^Ty$ .
The vector 2-norm is preserved under orthogonal transformations: $\|Qx\| = \|x\|$ .
The matrix 2-norm is preserved under orthogonal transformations: $\|QA\| = \|A\|$ .
The 2-norm of an orthogonal matrix is one: $\|Q\| = 1$ .

For further details, see the following Wikipedia pages

channelflow.org

User Tools

Site Tools

Math 753/853 Norms, inner products, and orthogonality

Inner product

2-norm

Orthogonal matrices

Page Tools