That depends very much on your axiomatization of $\sf ZF$.
Specifically, the axiom of pairing is a consequence of replacement, power set, and infinity (one could use "empty set" instead of infinity, but that would be redundant, since infinity implies the empty set exists directly).
If minimality is what your heart desires, then pairing is a theorem, not an axiom, and then I don't see why things work out in the finite case either.
But let's just put this aside, and let pairing be part of our system. Indeed, then in that case you need the axiom of replacement for infinite collections.
The classic example is $V_{\omega+\omega}$. We start with $V_0=\varnothing$, and for $n$, $V_{n+1}=\mathcal P(V_n)$. When we reach $\omega$, $V_\omega=\bigcup\{V_n\mid n<\omega\}$, and so we continue again with power sets and unions.
It is not hard to check that $V_{\omega+\omega}$ actually satisfies all the axioms of $\sf ZF$ except Replacement. Including pairing, just to be clear. But now consider the function given by $f(0)=V_\omega$ and $f(n+1)=\mathcal P(f(n))=V_{\omega+n+1}$. The range of $f$ is exactly $\{V_{\omega+n}\mid n<\omega\}$, and it is easy to see that this is not an element of $V_{\omega+\omega}$.
So indeed Replacement fails, and since pairing and union hold, it fails for infinite sets.