You can use these two observations:
- Under the given assumptions, you have $|X\setminus Y|\ge \aleph_0$.
- If $|A|\ge\aleph_0$, then $|A|+\aleph_0=|A|$.
We have $|X\setminus Y|\ge \aleph_0$. If $|X\setminus Y|<\aleph_0$ we would get that $|X|=|Y|+|X\setminus Y|\le \aleph_0+\aleph_0=\aleph_0$, a contradiction. So we have the opposite inequality - $|X\setminus Y|\ge\aleph_0$.
Notice that here we're using that cardinalities of any two sets are comparable - which is basically axiom of choice. (Of course, if you are working with some specific $X$ and $Y$, it is possible that you can immediately see that $|X\setminus Y|\ge\aleph_0$.)
You can also have a look at this post: Does $k+\aleph_0=\mathfrak{c}$ imply $k=\mathfrak{c}$ without the Axiom of Choice? (Especially if you want an argument avoiding AC.) Several questions linked there are close to yours. (Some of them for some specific sets $X$ and $Y$ rather than in full generality.)
$|A|+\aleph_0=|A|$ whenever $|A|\ge\aleph_0$. This is basically the "Hilbert hotel" argument. The inequality $|A|\ge\aleph_0$ means that we have a "copy" of $\mathbb N$ inside $A$. For this copy we can find a bijection with "two copies" of the same set. (This is the same proof as $\aleph_0+\aleph_0=\aleph_0$.)