I will outline a proof of the characterization already given by Geoffrey Trang.
Step 1. The primes of the form $x^2+y^2$ are $2$ and the primes $\equiv 1\pmod{4}$.
Trivially, a prime $\equiv 3\pmod{4}$ cannot be represented by $x^2+y^2$ and $2=1^2+1^2$.
If $p\equiv 1\pmod{4}$ the Legendre symbol $\left(\frac{-1}{p}\right)=(-1)^{\frac{p-1}{2}}$ gives that $-1$ is a quadratic residue in $\mathbb{F}_p$, so for some integer $a\leq\frac{p-1}{2}$ we have $a^2+1= kp$ with $k<\frac{p}{4}$. The fact that the norm in $\mathbb{Z}[i]$ is multiplicative leads to Lagrange's identity
$$ (a^2+b^2)(c^2+d^2) = (ad-bc)^2+(ac+bd)^2. $$
By Fermat's descent we are able to go from $a^2+1=kp$ to $a^2+b^2=p$: every prime $\equiv 1\pmod{4}$ can be written as a sum of two squares in a essentially unique way. By Lagrange's identity once again we have that any $n$ without prime factors of the form $4k+3$ can be written as a sum of two squares. The same holds if for every prime divisor $p$ of the form $4k+3$ we have $\nu_p(n)\equiv 0\pmod{2}$.
Step 2. If there is a prime divisor $p$ of the form $4k+3$, such that $\nu_p(n)$ is odd, $n$ cannot be written as a sum of two squares. In $\mathbb{F}_p$ we have that $-1$ is not a quadratic residue, hence $x^2+y^2\equiv 0\pmod{p}$ implies that both $x$ and $y$ are multiples of $p$. In turn this implies that $\nu_p(n)$ is even, contradiction.
Note. $21\equiv 1\pmod{4}$ but it cannot be written as a sum of two squares, since $\nu_3(21)$ is odd.
Something similar applies to $2021=43\cdot 47$.