Referencing tables when they are inside `onlyenv` (beamer)

Question

It seems that I cannot reference tables when they are placed inside onlyenv, but . This applies to both of \cref and the traditional \ref

The topic Table numbers in beamer is not an answer to this question because my question is asking about referencing a table, not inserting its number in a caption.

Here is the MWE. This should be run twice to get the real result.

\documentclass[aspectratio=169, xcolor={x11names}]{beamer}


\usecolortheme{seahorse}
\useoutertheme{miniframes}
\useinnertheme{inmargin}
\usefonttheme[onlylarge]{structurebold}
\setbeamertemplate{caption}[numbered]

% Command to change margins for a page wide
\makeatletter
\newcommand{\framePageWide}{
    \def\Gm@lmargin{5mm}%
    \def\Gm@rmargin{5mm}%
    \textwidth=\dimexpr\paperwidth-\Gm@lmargin-\Gm@rmargin\relax
    \hsize\textwidth
    \columnwidth\textwidth
    \setlength{\linewidth}{\textwidth}
    \hoffset=\dimexpr-\beamer@leftsidebar+\Gm@lmargin-0mm\relax
}
\makeatother
\newcommand{\framePageWideEnumI}{\setlength{\leftmargini}{10mm}}


    \usepackage{adjustbox}
    \usepackage{ragged2e}
    \usepackage{tabularx}
% BEGIN_FOLD

        \newcolumntype{C}{ >{\arraybackslash \Centering}X }

            \newcommand{\scalelinespace}[1]{
                \rule{0pt}{#1\normalbaselineskip}
                                        }

% END_FOLD

    \usepackage{booktabs}
    \usepackage{makecell}

    \usepackage{amsmath}
    \usepackage{amssymb, amsfonts}

    \usepackage{cleveref}
    \crefformat{table}{Table #1}


\begin{document}

{\framePageWide

\begin{frame}{Tables}

\begin{onlyenv}<1>

    \framePageWideEnumI

    \begin{itemize}
        \item Item 1
        \item Item 2
    \end{itemize}

\end{onlyenv}

\begin{onlyenv}<2>

    \begin{table}

        \begin{adjustbox}{minipage={0.5\textwidth}, valign=t}

            \caption{Time, in seconds, needed to calculate a single ${Z}_{bus}$ Element}
            \label{table:results2}

            \begin{tabularx}{\linewidth}{c C C}

                \toprule

                System Size & Our Method & Brown's Method
                \\
                \addlinespace

                200-bus & $4.3\times10^{-6}$ & $0.092$
                \\
                \Xhline{0.25mm}

                2383-bus & $3.8\times10^{-6}$ & $166.2$
                \\
                \Xhline{0.25mm}

                3120-bus & $3.2\times10^{-6}$ & $398.8$
                \\

                \bottomrule

            \end{tabularx}

        \end{adjustbox}
        \hfill
        \begin{adjustbox}{minipage={0.43\textwidth}, valign=t}

            From \cref{table:results2} we see that

            \begin{itemize}
                \item Our method is fast
                \item By contrast, Brown's becomes very slow when the system is large
            \end{itemize}

        \end{adjustbox}

    \end{table}

\end{onlyenv}


\end{frame}

}

\end{document}