The problem is triggered, if the redefinition of \mathchoice in package mathstyle meets LuaTeX's \mathstyle, from package mathstyle:
\def\mathchoice{%
\relax\ifcase\mathstyle
\expandafter\@firstoffour % Display
\or
\expandafter\@firstoffour % Cramped display
\or
\expandafter\@secondoffour % Text
\or
\expandafter\@secondoffour % Cramped text
\or
\expandafter\@thirdoffour % Script
\or
\expandafter\@thirdoffour % Cramped script
\else
\expandafter\@fourthoffour % (Cramped) Scriptscript
\fi
}
\ifcase expects a number and continues the expansion until a token is found, which does not contain to a number (non-digit). Thus the first \expandafter of case "0" for "display style" is called at the wrong time, when the number is still being read. A \relax stops that and should fix the issue:
\documentclass{beamer}
\usepackage{breqn}
\makeatletter
\def\mathchoice{%
\relax\ifcase\mathstyle\relax
\expandafter\@firstoffour % Display
\or
\expandafter\@firstoffour % Cramped display
\or
\expandafter\@secondoffour % Text
\or
\expandafter\@secondoffour % Cramped text
\or
\expandafter\@thirdoffour % Script
\or
\expandafter\@thirdoffour % Cramped script
\else
\expandafter\@fourthoffour % (Cramped) Scriptscript
\fi
}
\makeatother
\newcommand{\spam}{\textrm{Spam}}
\begin{document}
\begin{frame}
\centering
$\displaystyle \mathchoice{D}{T}{S}{s}$
\[ \spam \]
\end{frame}
\end{document}

\mathchoice. – Heiko Oberdiek Jul 04 '16 at 14:05\mathrmdoes not show the problem that\textrmdoes, so there is the answer to part 2 of my question. – gerrit Jul 04 '16 at 14:09luatextag was not yet there when I tried :) – samcarter_is_at_topanswers.xyz Jul 04 '16 at 15:14