0

I have difficulty solving the following minization problem: $$\textrm{minimize}\quad f(x,y)=x^TDx+y^TDy,$$ $$\textrm{subject to}\quad x^Tx=y^Ty=1, \textrm{and }x^Ty=0,$$ where $x,y\in \mathbb{R}^3,$ and $D=\left[\begin{array}{ccc}1 & 0 & 0\\0 & 2 & 0\\0 & 0 & 3\end{array}\right]$ is a diagonal matrix.

So we're looking for two orthonormal vectors, $x$ and $y$, in $\mathbb R^3$ which would minimize the sum of the quadratic forms, $x^TDx+y^TDy$.

If we're only considering $x^TDx$, it's well-known that the eigenvector of $D$ corresponding to the minimum eigenvalue is the solution, i.e. $x=e_1=[1\:0\:0]^T.$ Now, to minimize $x^TDx+y^TDy$, an intuitive attempt is to keep $x=e_1$, and choose $y$ to be the (orthonormal) eigenvector of $D$ corresponding the second smallest eigenvalue, i.e. $y=e_2=[0\:1\:0]^T$. This leads to $f(x,y)=1+2=3.$

However, I have difficulty proving that this is indeed the minimum value of this problem, and would appreciate it if someone know how we may prove this. (Better yet a proof that can be easily extended to higher dimensions, $\mathbb R^n$.) Thanks a lot!

syeh_106
  • 3,093

1 Answers1

1

We may rewrite the problem as follows: if we take $U$ to be the matrix with columns $x,y$, then this reads $$ \text{minimize} \quad f(U) = \operatorname{trace}(U^TDU)\\ \text{subject to} \quad U^TU = I $$ which corresponds to maximizing $\operatorname{trace}(U^T[-D]U)$ under the same constraint. Thus, we have reduced your problem to an instance of this problem. You'll find that the answers outline two different approaches here.

Community
  • 1
Ben Grossmann
  • 225,327
  • 1
    Thanks a lot for the pointers! Indeed, with variational characterization of eigenvalues, it's also not hard to show $\min \mathrm{tr}(U^TDU)$ is the sum of the $k$ smallest eigenvalues, and similarly $\max \mathrm{tr}(U^TDU)$ is the sum of the $k$ largest eigenvalues, with $U$ consisting of $k$ orthonormal columns. In fact, this is exactly Corollary 4.3.18 of Horn and Johnson's Matrix Analysis. – syeh_106 Feb 08 '17 at 04:48
  • @syeh_106 good find! I guess I never looked closely enough at the theorems in that chapter; it seems like Horn and Johnson has everything in it if you look close enough. – Ben Grossmann Feb 08 '17 at 05:37