This is an old revision of the document!
====== Math 753/853 Norms, inner products, and orthogonality ====== Ok, this is a big set of topics, and nothing I've found covers the topic at the right level of detail or depth. So, here's a summary of a few key points you should understand. ===Inner product=== The inner product of two vectors $x$ and $y$ \begin{equation*} x^Ty = \sum_{i=1}^m x_i y_i \end{equation*} where $x^T$ is the transpose of $x$. If $x^Ty = 0$, $x$ and $y$ are orthogonal. ===2-norm=== The 2-norm of a vector $x$ is defined as \begin{equation*} \|x\| = \sqrt{\sum_{i=1}^m x_i^2} \end{equation*} Note that $x^Tx = \|x\|^2$. The 2-norm of a matrix $A$ is defined as \begin{equation*} \|A\| = \sup_{x\neq0} \frac{\|Ax\|}{\|x\|} = \sup_{\|x\| = 1 } \|Ax\| \end{equation*} You can think of $\|A\|$ as the maximum amplification factor in length that can occur under the map $x \rightarrow Ax$. ===Orthogonal matrices=== A matrix $Q$ is an orthogonal matrix if its inverse is its transpose: $Q^T Q = I$. The columns of an orthogonal matrix are a set of orthogonal vectors. Key properties of orthogonal matrices: * The inner product is preserved under orthogonal transformations: $(Qx)^T(Qy) = x^Ty$. * The vector 2-norm is preserved under orthogonal transformations: $\|Qx\| = \|x\|$. * The matrix 2-norm is preserved under orthogonal transformations: $\|QA\| = \|A\|$. * The 2-norm of an orthogonal matrix is one: $\|Q\| = 1$. For further details, see the following Wikipedia pages * [[https://en.wikipedia.org/wiki/Matrix_norm | Matrix norms ]] * [[https://en.wikipedia.org/wiki/Dot_product | Inner products ]] * [[https://en.wikipedia.org/wiki/Transpose | Transpose ]] * [[https://en.wikipedia.org/wiki/Orthogonal_matrix | Orthogonal matrices ]]
