abraces can be used for this - swapping/mixing ofthe brace directions:

\documentclass{article}
\usepackage{abraces,mathtools}% http://ctan.org/pkg/{abraces,mathtools}
\begin{document}
\[
\setbox9=\hbox{$(X \circ X)$}
\operatorname{min} \big|
\mathrlap{\hspace{.5\wd9}\mathclap{\aunderbrace[L1U1R]{\scriptstyle\phantom{\text{Hadamard product}}}_{\text{Hadamard product}}}}
(X \circ X) - (1 \ 2 \ 3 \ \cdots \ M )\big|^2
\]
\[
\operatorname{min} \big|
\underbrace{(X \circ X)}_{\mathclap{\text{Hadamard product}}}
- (1 \ 2 \ 3 \ \cdots \ M )\big|^2
\]
\end{document}
The use of \box9 is just for finding the correct width of (X \circ X). That is, some box movement is required in order to place the \aunderbrace at the correct location. The second option looks better though.
You could use an \overbrace as well. And, using \big (and friends) instead of \left...\right allows for a better appearance in terms of the absolute delimiters.
\underbrace– azetina Jan 29 '13 at 19:43