To understand the behaviour, the following quick example is useful
\[\color{red}\frac{2}{a}\]
\[{\color{red}\frac{2}{a}}\]
The first line shows the bigger spacing, the second line is surrounded by an additional group and has tighter spacing. To cite the related comment of David Carlisle
see the footnote in the documentation of the color package, color will sometimes affect spacing. the footnote is there basically to explain that it's not my fault (texdoc grfguide page 6 :-)
\everydisplay{\color{blue}} clearly adds a \color to the math display and triggers this behaviour. \setbeamercolor{math text displayed}{fg=red} also triggers it, as normally the math is colour is not set and \color is will only be added to the math display if the colour is not empty (see https://github.com/josephwright/beamer/issues/177)
Solution 1
To get a uniform behaviour and prevent the jumping between the slides, you can set the math colour at the beginning of the document. This will give all math displays the larger spacing.
\documentclass{beamer}
\setbeamercolor{math text displayed}{parent=normal text}
\begin{document}
\begin{frame}
\only<2>{\everydisplay{\color{blue}}}
\[ \frac{2}{a} \]
\[ \frac{2}{a} \]
\end{frame}
\end{document}
Solution 2
Or if you prefer the tight spacing, the following is based on https://tex.stackexchange.com/a/91551/36296
\documentclass{beamer}
\setbeamercolor{mymath}{parent=normal text}
\def\foo#1$$#2!!{\def\mathdisplay##1{#1$$\begingroup\usebeamercolor[fg]{mymath}#2}}
\expandafter\foo\mathdisplay{#1}!!
\long\def\foo#1$$#2!!{\def\endmathdisplay##1{#1\endgroup$$#2}}
\expandafter\foo\endmathdisplay{#1}!!
\begin{document}
\begin{frame}
\only<2>{\setbeamercolor{mymath}{fg=red}}
\[ \frac{2}{a} \]
\[ \frac{2}{a} \]
\end{frame}
\end{document}
\setbeamercolor{math text displayed}{fg=red}has the same problem – samcarter_is_at_topanswers.xyz Jan 13 '17 at 19:57\[\textcolor<2>{red}{\frac{2}{a}}\]%be an acceptable workaround? – samcarter_is_at_topanswers.xyz Jan 13 '17 at 20:02\setbeamercolor{math text displayed}{fg=red}seems to change the spacing to the new one but it remains the same through the slides. – Kurzd Jan 13 '17 at 20:06texdoc grfguidepage 6 :-) (@samcarter) – David Carlisle Jan 13 '17 at 21:12