0

Suppose $Rank(A)=r,A{\in}R^{m \times n}$, $v_1,v_2,...,v_r{\in}R^n$ forms an orthonormal basis of $C(A^T)$(i.e. row space of A) and are eigenvectors of $A^TA$.

Now, take $\sigma_1u_1=Av_1,\sigma_2u_2=Av_2,...,\sigma_ru_r=Av_r;\ \sigma_1,\sigma_2,...,\sigma_r$ are positive scaling factors and $u_1,u_2,...,u_r{\in}R^m$ are unit vectors in $C(A)$(column space of A).

Now,we have $A(v_1,v_2,...,v_r)=(u_1,u_2,...,u_r)\begin{bmatrix}\sigma_1& 0 &\cdots &0 \\ 0& \sigma_2 & \cdots &0 \\ \vdots &\vdots &\ddots &\vdots \\0&0&\cdots&\sigma_r\end{bmatrix}$. And denote this formula as $AV=U\Sigma,V{\in}R^{n\times r},U{\in}R^{m\times r},\Sigma$ is a r$\times$r diagonal matrix.

My problem is how to show $u_1,u_2,...,u_r$ is an orthonormal basis of $C(A)$? E.g. $u_1^Tu_2=0$ when take $\sigma_1u_1=Av_1,\sigma_2u_2=Av_2$.

Above thread comes from MIT's opencourse 18.06 linear algebra lecture 29

Finley
  • 1,133
  • The vectors $v_i$ must be chosen in a special way. – littleO Aug 30 '17 at 02:15
  • @littleO : Can you state the trick concretely? – Finley Aug 30 '17 at 02:32
  • @Finley This is a very strange way to present the idea of finding an SVD. Where is this question coming from; that is, what have you learned/read most recently? Have you tried reading a textbook and finding the SVD with the method outline there? – Ben Grossmann Aug 30 '17 at 02:39
  • @Finley how about this: are you at least aware that the $\sigma_i$ must be the square roots of the eigenvalues of $A^TA$? – Ben Grossmann Aug 30 '17 at 02:42
  • Here's a relevant thread that explains one way of understanding the SVD: https://math.stackexchange.com/questions/1737637/understanding-a-derivation-of-the-svd – littleO Aug 30 '17 at 02:42
  • @Omnomnomnom : From MIT's opencourse18.06 linear algebra lecture 29 – Finley Aug 30 '17 at 02:43
  • In Strang's approach, the vectors $v_i$ should be chosen to be an orthonormal basis of eigenvectors for $A^T A$. – littleO Aug 30 '17 at 02:45
  • @Omnomnomnom : Yeah, I can understand what you say is a fact when any matrix $A{\in}R^{m\times n}$ can be decomposed as $A=U\Sigma V^T$ – Finley Aug 30 '17 at 02:48
  • @Finley It's not clear how your approach fit into what Strang is doing. For instance, you don't start by saying that the $v_i$ are eigenvectors of $A^TA$. – Ben Grossmann Aug 30 '17 at 02:57
  • @Omnomnomnom : Yep! I miss this point. I get it: $\sigma_1 \sigma_2 u_1^Tu_2=v_1^TA^TAv_2=\lambda_2 v_1^Tv_2=0$ with $A^TAv_2=\lambda_2 v_2$ for $v_1,v_2$ are othorgonal eigenvectors of $A^TA$ – Finley Aug 30 '17 at 03:04

1 Answers1

1

Assuming everything in the problem statement, ... \begin{align*} u_i^\mathrm{T} u_j &= \frac{1}{\sigma_i \sigma_j} \sigma_i \sigma_j u_i^\mathrm{T} u_j \\ &= \frac{1}{\sigma_i \sigma_j} \sigma_i u_i^\mathrm{T} \sigma_j u_j \\ &= \frac{1}{\sigma_i \sigma_j} (\sigma_i u_i)^\mathrm{T} (\sigma_j u_j) \\ &= \frac{1}{\sigma_i \sigma_j} (A v_i)^\mathrm{T} (A v_j) \\ &= \frac{1}{\sigma_i \sigma_j} v_i^\mathrm{T} (A^\mathrm{T} A) v_j \\ &= \frac{1}{\sigma_i \sigma_j} v_i^\mathrm{T} \varepsilon_j v_j \\ &= \frac{1}{\sigma_i \sigma_j} \varepsilon_j v_i^\mathrm{T} v_j \\ &= \frac{1}{\sigma_i \sigma_j} \varepsilon_j 0 \\ &= 0 \text{,} \end{align*} where $\varepsilon_j$ is the eigenvalue of $(A^\mathrm{T} A)$ associated to $v_j$.

Eric Towers
  • 67,037
  • In fact, the key is to understand r eigenvectors(given that $Rank(A)=r$) corresponding to nonzero eigenvalues of $A^TA$ exactly right forms an orthonormal basis of $C(A^T)$ . And notice $A^T(Av/\lambda) =v$ holds for any nonzero eigenvalue $\lambda$. – Finley Aug 30 '17 at 06:53