First, consider $Y = \Bbb{CP}^\infty \times S^1$. Its cohomology ring over $\Bbb Z/2 =: \Bbb F$ is $\Bbb F[a] \otimes \Lambda[c]$ where $|a| = 2$ and $|c| = 1$. The Bockstein homomorphism can be seen to be trivial (as you say, cellular cohomology does the trick), and thus so is $\text{Sq}^1$. Now $\text{Sq}^2(ac) = \text{Sq}^2(a)c + \text{Sq}^1(a)\text{Sq}^1(c) + a\text{Sq}^2(c)$; because $|c| = 1$ the last term vanishes, and we have already discussed why the middle term vanishes; because $|a| = 2$, $\text{Sq}^2(a) = a^2$. Hence $\text{Sq}^2(ac) = a^2c$. If $X$ is your second space, the quotient map $p: Y \to X$ induces an injection $H^*(X) \to H^*(Y)$, sending $p^*(a) = a$ and $p^*(b) = ac$. Hence by naturality of Steenrod squares $\text{Sq}^2(b) = ab$.
Now consider $\Bbb{CP}^\infty \times S^3$. We want to calculate $\text{Sq}^2(b)$ here. The inclusion $S^2 \times S^3 \hookrightarrow \Bbb{CP}^\infty \times S^3$ induces an isomorphism on cohomology in degrees $\leq 5$, so by naturality we just need to understand what $\text{Sq}^2(b)$ is in the first space.
A product $X \times Y$ of spaces with trivial Steenrod squares has trivial Steenrod squares. First note that because Steenrod squares are additive it suffices to see how the squares behave on elements in $H^i(X) \otimes H^j(Y) \subset H^{i+j}(X \times Y)$; and then by the Cartan formula it suffices inductively to see they vanish on $H^i(X)$ and $H^i(Y) \subset H^i(X \times Y)$. Now invoke the projection maps $\pi: X \times Y \to X$; by the vanishing of the Steenrod squares on $X$ and naturality, they vanish on these primitive classes in $X \times Y$.
Alternatively, we know that the Steenrod square commutes with the suspension isomorphism $H^*(S^2 \times S^3) \to H^{*+1}(\Sigma(S^2 \times S^3))$. As seen here, $\Sigma(X \times Y)$ is homotopy equivalent to $\Sigma X \vee \Sigma Y \vee \Sigma(X \wedge Y)$. In this case, this says that $\Sigma(S^2 \times S^3)$ is homotopy equivalent to $S^3 \vee S^4 \vee S^6$. This space has vanishing Steenrod squares, as does any wedge of spaces with vanishing Steenrod squares. (This is again by naturality, because we have projection maps $X \vee Y \to X$.)
Here's an alternate approach that Hatcher definitely did not intend, invoking characteristic classes of smooth manifolds; you can learn about these in Milnor and Stasheff's book on the subject.
Again we want to calculate $\text{Sq}^2(b)$ on $S^2 \times S^3$. Now, on a closed connected manifold, there are cohomology classes called the Wu classes, such that for $x \in H^{n-k}(M)$, $\text{Sq}^k(x) = x \smile v_k$. They are related to the Stiefel-Whitney classes of the manifold in that $\text{Sq}(v) = w$. Now, $S^2 \times S^3$ is parallelizable, so all of its Stiefel-Whitney classes vanish. This implies that $v=0$, and hence all of the Steenrod squares on this manifold (or any parallelizable manifold) vanish. Whence $\text{Sq}^2(b) = 0$, as desired.
In any case, we have now seen that $\text{Sq}^2(H^3(X)) \neq 0$, but $\text{Sq}^2(H^3(\Bbb{CP}^\infty \times S^3)) = 0$, proving the desired claim.