0

I was watching Singular Value Decomposition Lecture by Gilbert Strang. He takes two orthonormal vectors $v_1$, $v_2$. Let $\sigma_1$$u_1$=A$v_1$ and $\sigma_2$$u_2$=A$v_2$. He takes $u_1$ is orthogonal to $u_2$. How? How to prove this?

Edit1: Adding link to video lecture https://www.youtube.com/watch?v=Nx0lRBaXoz4 (Start from 3:30 mins)

Nagabhushan S N
  • 796

2 Answers2

0

Recall that the SVD is given by $ A = U \Sigma V ^ T $, hence $ A ^ T A = ( U \Sigma V ^ T) (U \Sigma V ^ T)^T = U \Sigma ^ 2U ^T$. Namely, the columns of $U$ are the eigenvectors of $A ^ T A$, and as $A^TA$ is symmetric it is can be diagonilzed by an orthogonal matrix ($U$).

V. Vancak
  • 16,444
  • I ran into the problem while deriving SVD! – Nagabhushan S N May 13 '18 at 22:08
  • $u_1$ and $u_2$ are the first and the second column of $U$, which is orthogonal, hence by definition $u_1 ^ T u_2 = 0$. And the proof (that $U$ is orthogonal follows from my answer). – V. Vancak May 13 '18 at 22:12
  • How do you say $U$ is orthogonal? In your answer, you've used SVD. But as I said before, Gilbert Strang uses the fact that $U$ is orthogonal to prove SVD. So, there should be another way to prove that $U$ is orthogonal without using SVD. I'm stuck there. – Nagabhushan S N May 14 '18 at 14:29
0

I found an explanation here and here. It explains that we choose $v_1$ such that $Av_1$ is maximum. Then we choose $v_2$ such that $v_2 \perp v_1$ and $Av_2$ is maximum. In that case, the above referred answers prove that $$Av_1 \perp Av_2$$ We choose $$u_1=Av_1/||Av_1||$$ $$u_2=Av_2/||Av_2||$$ Hence $$ u_1 \perp u_2$$

I guess Gilbert Strang omitted/missed this explanation in that lecture.

Nagabhushan S N
  • 796