How to make Tufte sidenotes behave properly in tcolorbox environments?

Question

Marginnotes (sidenotes), and therefore citations in the tufte-book class don't play nice with the theorem boxes in tcolorbox. I've tried to rewrite the citation and marginnote commands in tufte using a macro that detects the current environment (Relevant Links Here) but I haven't been able to make it work.

MWE:

\documentclass[justified, nofonts, notitlepage, openany, debug]{tufte-book}
\usepackage{tcolorbox}
\tcbuselibrary{skins}
\tcbuselibrary{breakable}
\tcbuselibrary{theorems}
\newtcbtheorem{theorem}{Theorem}{parbox=false, breakable, sharp corners, skin=enhanced, frame hidden, colback=white, coltitle=white, colbacktitle=black, adjusted title, fonttitle=\bfseries}{th}
\def\bibsection{~\ \section*{\refname}}
\begin{document}
\begin{theorem}{Pythagoras\cite{kontsevich-manin}}
    $$a^2 + b^2 = c^2$$\marginnote{Lorem ipsum et cetera}
\end{theorem}
\marginnote{Regular marginnote}
Here's a regular citation to illustrate expected behavior:\cite{witten}
\begin{theorem}{\marginnote{The quick brown fox or something}}
    $$d^2 + e^2 = f^2$$\cite{hartshorne}
\end{theorem}
\bibliographystyle{plainnat}
\bibliography{sample}
\end{document}

Above code produces the following:

Expected (desired) behavior is for citations to print to a marginnote outside of the tcolorbox, and to maintain numbering with the rest of the document.

Edit: I attempted fixes in the vein of this question, to save the sidenotes from a theorem environment and place them after the end of the environment, but was unable to get it working. Another strategy is to redefine \cite and \marginnote to behave one way if not inside a theorem box, and to behave differently if inside the theorem box.