Ok, this is a big set of topics, and nothing I've found covers the topic at the right level of detail or depth. So, here's a summary of a few key points you should understand. These were spelled out in detail during lecture.
The inner product of two vectors and
where is the transpose of . If , and are orthogonal.
The 2-norm of a vector is defined as
Note that .
The 2-norm of a matrix is defined as
You can think of as the maximum amplification factor in length that can occur under the map .
A matrix is an orthogonal matrix if its inverse is its transpose: . The columns of an orthogonal matrix are a set of orthogonal vectors.
Key properties of orthogonal matrices:
For further details, see the following Wikipedia pages