Let's look at your input:
{\textbf{cos(W_M, V_M)}\overset{\Delta}{=}\frac{W_{M_i}.V_{M_j}}{(\left\|W_{M_i} \right\|\left\|V_{M_j}\right\|)}}
- The outer braces do nothing at all, so it's best to remove them
\textbf just produces a warning and does nothing
- “cos” should be
\cos
\overset{\Delta}{=} is good, but probably you want \triangleq
- The inner (or scalar) product should not be denoted by a (low) period, but rather with a centered dot
- The parentheses
(...) are not needed in the denominator and make for harder reading1
\left and \right in your code do harm, rather than being helpful
Points 1–4 and 6 can be addressed very easily. For point 5, use \cdot.
Let's come to the main point of your question. For the norm of vectors, use
\lVert <vector> \rVert
It's generally not needed (nor advisable) to use \left\lVert and \right\rVert, because the automatically sized fences might be of different size or too tall anyway.2
Note, instead, \, between the two norms to help readers in distinguishing between them.3
\documentclass{article}
\usepackage{amsmath}
\usepackage{amssymb}
\begin{document}
\begin{equation}
\cos(W_M, V_M)
\triangleq
\frac{W_{M_i} \cdot V_{M_j}}{\lVert W_{M_i} \rVert,\lVert V_{M_j}\rVert}
\end{equation}
\end{document}

Footnotes
1 With parentheses in the denominator, a reader might think that there's something missing. Avoid redundant parentheses.
2 Here's what you get from adding \left and \right in front of \lVert and \rVert

Can you see the difference, which is caused by the fact that “j” descends below the baseline? There is no need that the fences completely cover what's between them, particularly if it's just a descender or an accent that's not covered.
3 Without \, the output would be

which requires the reader to count the bars.