This is an application of Euler's criterion. If $p$ is an odd prime and $a$ is a number coprime with $p$, then
\begin{equation}
a^{\tfrac{p-1}{2}}\equiv
\begin{cases}
1 & \quad \text{if there exists an integer } x \text{ such that } x^2\equiv a \mod p \\
-1 & \quad \text{if there is no such integer}
\end{cases}
\end{equation}
Two elementary proofs of this can be found on wiki: https://en.wikipedia.org/wiki/Euler%27s_criterion
Let us apply this with $a=p-1$, which is obviously coprime with $p$.
If $p=4k+1$, then $(p-1)^{\tfrac{p-1}{2}}=(p-1)^{2k}\equiv (-1)^{2k}\equiv 1\mod p$, so by Euler's criterion, there exists some integer $x$ such that $x^2\equiv p-1\mod p$, meaning that $x^2+1=py$ for some integer $y$.
If $p=4k+3$, then $(p-1)^{\tfrac{p-1}{2}}=(p-1)^{2k+1} \equiv (-1)^{2k+1} \equiv -1\mod p$, so by Euler's criterion, there can exist no integer $x$ such that $x^2+1\equiv 0\mod p$.