1

How can I get intuition/proof for the fact that the $L_2$ norm of $X X^T$ and $X^T X$ are always the same for any real matrix $X$?

Sunny88
  • 1,069
  • What have you tried? Have you, for instance, considered the spectra (the collections of eigenvalues) for your two products? – Eric Towers Nov 20 '15 at 06:45
  • @EricTowers I see, so the L2 norm of a matrix is equal to the L2 norm of vector of its singular values. And the singular values are the same for the two products as can be shown by replacing X by its SVD in the products. – Sunny88 Nov 20 '15 at 07:22
  • Actually, you're only interested in the largest singular value. See http://math.stackexchange.com/questions/586663/why-does-spectral-norm-equal-the-largest-singular-value – Eric Towers Nov 20 '15 at 12:58
  • @EricTowers That link is about spectral norm, but I am asking about L2 norm. Why is it relevant? – Sunny88 Nov 21 '15 at 12:06

2 Answers2

2

In case you mean by $L^2$ norm of a matrix the sum of squares of all entries, which I denote by $\|\cdot\|_{F}$ use that $\|A\|_{F}=\text{tr}(A^\top A)$ and use rules of permuting arguments within the trace.

Bananach
  • 7,934
0

Note that since $X^TX$ and $XX^T$ are symmetric matrices, $$\|X^TX\|_2 = \lambda_{max}(X^TX)$$ $$\|XX^T\|_2 = \lambda_{max}(XX^T)$$ So to prove that the two norms are equivalent simply boils down to proving that the largest eigenvalues of $X^TX$ and $XX^T$ are equal.

Let $X^TXv = \lambda v.$ Then clearly $XX^TXv = X\lambda v \rightarrow (XX^T)Xv = \lambda Xv.$ Thus $X^TX$ and $XX^T$ have the same eigenvalues which means they also have the same maximum eigenvalue (and $\|\cdot\|_2$ norm).

sfTurpin1990
  • 85
  • Your equations are for spectral norm, but my question is about $L_2$ norm (https://en.wikipedia.org/wiki/Matrix_norm#L2.2C1_norm). $L_2$ norm is not equal to maximum eigenvalue. – Sunny88 Nov 26 '15 at 16:28