The code:
vec1 = {{-1}, {0}, {1}} // MatrixForm
vec2 = {{-1}, {1}, {0}} // MatrixForm
p[v2_, v1_] := v1 (v1.v2)/Sqrt[(v1.v1) (v2.v2)];
u2 = vec2 - p[vec2, vec1]
The output:
Mathematica won't calculate the dot product in the output. I've tried Simplify, FullSimplify, Reduce, and MatrixForm. The last two were total guesses.
