Let $A \in \mathbb{R}^{m \times n}$ with $m \ge n$ and $b \in \mathbb{R}^n$. Then $x^* \in \mathbb{R}^n$ solves the problem
$$min_{x \in \mathbb{R}^n} \|Ax-b\|_2 \text{ for } $$
iff $x^*$ is a solution of the linear equation system $$A^TAx = A^Tb.$$
My idea would be to use the fact that:
Let $Q \in \mathbb{R}^{n \times n}$ be symmetric and positive semidefinit, $c \in \mathbb{R}^n$ and $\gamma \in \mathbb{R}$. Then $x^* \in \mathbb{R}^n$ is a minimum of the quadratic function $$f(x) = \frac{1}{2} x^tQx+c^tx+ \gamma$$ on $\mathbb{R}^n$ iff $x^*$ is the solution of the linear equation system $$Qx+c = 0.$$
We note that $A^TA$ is symmetric and positive semidefinite, so the above theorem fits well. Now I need to get $A^T$ into the term $\|Ax-b\|_2$, but how could I do that?
Could you help me?
