With a current latex-dev (What is "latex-dev"?) you can use \mathcolor. (I also use $ $ for the math, that is less fragile).
\mathcolor doesn't work yet with xcolor, but will soon:
\documentclass[border=2pt]{standalone}
\usepackage[dvipsnames]{color}
\usepackage[outline]{contour}\contourlength{1pt}
\newcommand{\border}[1]{\contour{red}{#1}}
\begin{document}
\border{(F_{!!\infty})}
\border{$\mathcolor{white}{F}{!!\infty}$}
\border{$\textcolor{white}{F}{!!\infty}$}
\end{document}

Update
contour disables colors locally and this confused \mathcolor. I think this here would be better:
\documentclass[border=2pt]{standalone}
\usepackage{color}
\usepackage[outline]{contour}\contourlength{1pt}
\newcommand{\border}[1]{\contour{red}{#1}}
\makeatletter
\def\con@coloroff{%
\def\set@color{%
\c@lor@special\m@ne
{color push \current@color}\aftergroup\reset@color}%
\def\reset@color{\c@lor@special\m@ne{color pop}}}
\makeatother
\begin{document}
\border{\(F_{\!\!\infty}\)}
\border{\(\mathcolor{white}{F}_{\!\!\infty}\)}
\border{$\mathcolor{white}{F}_{\!\!\infty}$}
\end{document}
\textcolor{}{}throughout my entire document. – Atcold Apr 11 '22 at 16:30