This is not a question about set theory specifically, but lets talk about ZFC just for concreteness
Suppose we have a sentence $\phi$ in the language of ZFC, and a proof that either $(\mathrm{ZFC} \vdash \phi) \vee (\mathrm{ZFC} \vdash \neg\phi)$. We believe $\mathrm{ZFC} \vdash \phi$, but actually demonstrating this is very difficult. So after a lot of head-banging, we get crafty: instead of trying to demonstrate $\mathrm{ZFC} \vdash \phi$, we instead properly extend our theory with a new axiom, say $\mathrm{GCH}$. Note this is not a conservative extension. We demonstrate $\mathrm{ZFC}+\mathrm{GCH} \vdash \phi,$ and thereby conclude that $\mathrm{ZFC} \vdash \phi$.
Has this sort of approach ever been used in practice?