Newbie in LaTeX here. On this the line I'm getting "missing $" error but I don't know where exactly it wants me to put it:
$g_2 $= $N_2${$f_2$ - (\underbrace($\int_a^b g_1^{*}f_2dx$)_{$g_1^t\cdot f_2$} $g_1$}\\
Someone help?
Newbie in LaTeX here. On this the line I'm getting "missing $" error but I don't know where exactly it wants me to put it:
$g_2 $= $N_2${$f_2$ - (\underbrace($\int_a^b g_1^{*}f_2dx$)_{$g_1^t\cdot f_2$} $g_1$}\\
Someone help?
For the version from your comment, without \underbrace. General rule: open math mode and stay within it through the full formula, if possible.
\documentclass{article}
\begin{document}
$g_2 = N_2{f_2 - (\int_a^b g_1^{}f_2\,dx)g_1^t\cdot f_2g_1}$
\end{document}

Probably you want a different equality, but the general rule is clear: one opening and one closing math sign.
I'm a bit confused about what you're trying to produce here, but you've definitely split up a math expression that shouldn't be split up.
Your use of square brackets and curly brackets it a bit confusing.
Try something like (my best guess at what you meant, though I find it a bit odd):
\documentclass{article}
\pagestyle{empty}
\begin{document}
\[
g_2 = N_2 f_2 - \bigg(\underbrace{\int_a^b g_1^{*}f_2dx}_{g_1^t\cdot f_2} g_1\bigg)
\]
\end{document}

\underbrace takes two arguments and should be written as
$\underbrace{<math stuff>}{<content for brace>}$
Or perhaps you meant:
\documentclass{article}
\pagestyle{empty}
\begin{document}
\[
g_2 = N_2 \bigg\{ f_2 - \underbrace{\int_a^b g_1^{*}f_2dx}_{g_1^t\cdot f_2} g_1 \bigg\}
\]
\end{document}

} or perhaps you have them improperly balanced.
– A.Ellett
Dec 20 '13 at 04:36
\{, \}, as in the example. Without it TeX thinks that they are grouping braces.
– Przemysław Scherwentke
Dec 20 '13 at 04:40