EDIT: Following Theo's comment, the equivalence holds since one can (must) rewrite $1/a$ as $(1+23k)/a$.
Provided that $$\frac{1}{25} \equiv \frac{1}{2}\pmod {23}$$ is true, why can I not rewrite it like usually possible?
$$\frac{1}{25}=\frac{1}{2}+23k ~\text{ for }~k \in \mathbb Z$$
$$\implies 2=25+50\cdot23k, \text{which is impossible.}$$
I guess that the factor 50 somehow may be ignored here, but I don't understand how (if at all). What's the clue here?
You have to be careful when considering fractions in modular arithmetic. Was does it mean to write $\frac{1}{25} \equiv \frac{1}{2} \mod 23$ ?
Here are two possible interpretations :
You may see that as a relation in the quotient ring $\mathbb{Z}/23\mathbb{Z}$ of integers modulo $23$. Here, $\frac{1}{25}$ is just a notation for the multiplicative inverse of the class of $25$ modulo 23 (that is the class of 12). In that point of view, this is not the class of the rational number $\frac{1}{25}$, because this is not even defined !
You may see that as a relation in the quotient of a bigger ring than $\mathbb{Z}$ that contains $\frac{1}{25}$. Indeed, denote
$$\mathbb{Z}_{(23)} = \left\{\frac{a}{b} \in \mathbb{Q}, \, b \textrm{ coprime to } 23\right\}$$
Then you can check this is a ring, and it contains $\frac{1}{2}$ and $\frac{1}{25}$. The good news is the quotient ring $\mathbb{Z}_{(23)}/23\mathbb{Z}_{(23)}$ is actually isomorphic to $\mathbb{Z}/23\mathbb{Z}$. This allows you to talk about the class of $\frac{1}{25}$ modulo $23$. There's a small price to pay however : the relation $\frac{1}{25} \equiv \frac{1}{2} \mod 23$ translates to $\frac{1}{25} = \frac{1}{2} + 23 k$ with $k$ being no longer an integer but $k \in \mathbb{Z}_{(23)}$.
