' is defined in latex.ltx as follows
\def\active@math@prime{^\bgroup\prim@s}
{\catcode`\'=\active \global\let'\active@math@prime}
\def\prim@s{%
\prime\futurelet\@let@token\pr@m@s}
\def\pr@m@s{%
\ifx'\@let@token
\expandafter\pr@@@s
\else
\ifx^\@let@token
\expandafter\expandafter\expandafter\pr@@@t
\else
\egroup
\fi
\fi}
\def\pr@@@s#1{\prim@s}
\def\pr@@@t#1#2{#2\egroup}
I can see that the catcode of ' is changed to \active in a group.
When the group ends, the catcode of ' falls back to 12 (other).
So how could it possibly be that $f'$ means $f\active@math@prime$?
I also tested
\catcode`!=12
{\catcode`!=\active \global\let!=\active@math@prime}
$f!!!$
It does not give the same result as $f'''$.
What happens to '?

!nor did I need a back up of'. But that\mathcodeis (probably) exactly what I am missing. – Symbol 1 Nov 20 '19 at 21:32\mathcodeset to"8000is not really active: only when TeX is processing math codes it gets replaced by its definition as if it were active. If you try$\edef\test{'}\show\test$, you'll see that no expansion is done to'even if TeX is in math mode. Also, only characters with category code 11 or 12 can be math active. – egreg Nov 20 '19 at 21:46$\def'{what?}$. And I got inaccessible. Now the rule is clear. – Symbol 1 Nov 20 '19 at 21:52