Since tocloft and memoir are by the same author, most tocloft solutions will work with memoir, but with slightly changed names. Roughly, where tocloft uses abbreviated sectioning names like chap, sec, subsec etc., memoir usually use the full version of the name. So here's the approach shown here adapted to memoir. You need to apply the patch to each sectioning level that you want it to work at. In your case, since you don't want the page number displayed at all, there's no need to make the code use \textcolor{} at all, but I've left that option available in the commented code.
Since the original code (which is not originally mine) effectively uses the displayed page number, it fails if the numbering is not arabic, which means that if you put \frontmatter in the document, it will fail, since the page numbering is set to roman. I've now fixed this and cleaned up the code substantially to deal with repeated roman or arabic page numbers.
\documentclass{memoir}
\settocdepth{subsection}
\setsecnumdepth{subsection}
\ExplSyntaxOn
% this function from https://tex.stackexchange.com/a/427559/
\prg_new_protected_conditional:Npnn \if_is_int:n #1 { T, F, TF }
{
\regex_match:nnTF { ^[\+\-]?[\d]+$ } {#1}% $
{ \prg_return_true: }
{ \prg_return_false: }
}
\int_new:N\l_clrize_svpage_int
\int_set:Nn\l_clrize_svpage_int{0}
\NewDocumentCommand{\resetsvpage}{}{\int_set:Nn\l_clrize_svpage_int{0}}
\cs_new:Npn\clrize#1{
\if_is_int:nTF{#1}
{\int_set:Nn\l_tmpa_int{#1}}
{\int_set:Nn\l_tmpa_int{\int_from_roman:n{#1}}}
\int_compare:nNnTF{\l_tmpa_int}={\l_clrize_svpage_int}
{}
% Instead of the previous line, use the next line
% to change the colour of the page number instead
% {\textcolor{white}{#1}}
{#1\int_gset:Nn \l_clrize_svpage_int {\l_tmpa_int}}
}
\ExplSyntaxOff
\usepackage{xpatch}
\usepackage{xcolor}
\xpatchcmd\cftchapterfillnum{#1}{\clrize{#1}}{}{}
\xpatchcmd\cftsectionfillnum{#1}{\clrize{#1}}{}{}
\xpatchcmd\cftsubsectionfillnum{#1}{\clrize{#1}}{}{}
\pretocmd{\mainmatter}{\addtocontents{toc}{\resetsvpage}}
\makeatother
\begin{document}
\frontmatter
\tableofcontents
\listoffigures
\mainmatter
\chapter{A chapter}
\section{Test}
\clearpage
\section{test}
\subsection{test}
\subsection{test}
\section{test}
\clearpage
\subsubsection{test}
\section{test}
\subsection{test}
\section{test}
\end{document}

subsectionis followed bysection? https://imgur.com/a/16YFDog – ezgranet Nov 18 '23 at 22:53\frontmatteris set? Triggers missing number errors withfrontmattercontent – ezgranet Nov 20 '23 at 02:16\frontmatterand\mainmatterpage numbers of the contents. – Alan Munn Nov 20 '23 at 17:15