We will ignore most of your assumptions, let $d = d_1$ be a number and assume that for $1 \le k \le x$ we have that $k$ is a square modulo $d$ if and only if $k$ is a square in $\mathbb{N}$. We will show that $d$ is at least $x^{2-o(1)}$ (with Burgess bound we should be able to improve $2$ to something slightly bigger, probably up to $4$).
We will assume that $d$ is smaller than that and simply show that the number of quadratic residues modulo $d$ up to $x$ is at least $x^{1-o(1)}$, which is bigger than the number of squares (which is at most $\sqrt{x}+1$).
Let us for simplicity assume that $d$ is squarefree (there are some annoying but not insurmountable technical issues otherwise). Consider all quadratic characters $\chi_1, \ldots, \chi_m$ modulo $d$, where $m = 2^{\omega(d)}$ and $\chi_1$ is a trivial character: $\chi_1(k) \equiv 1$ if $(k, d) = 1$ and $0$ otherwise. Consider the sum
$$f(k) = \chi_1(k) + \ldots + \chi_m(k).$$
I claim that if $k$ is not a square modulo $d$ then this sum is $0$. Indeed, this sum can be expressed as a product over primes $p$ dividing $d$ of $(1+\chi_p(k))$ and $\chi_p(k) = -1$ precisely if $k$ is a non-residue modulo $p$. So, we have $f(k) = 0$ if $k$ is a non-residue modulo $d$ and $f(k) \le 2^{\omega(d)}$ in general. Thus, the number of residues from $1$ to $x$ is at least
$$\frac{1}{2^{\omega(d)}}\sum_{k = 1}^x f(k) = \frac{1}{2^{\omega(d)}}\sum_{s=1}^m \sum_{k=1}^x \chi_s(k).$$
The sum for $s = 1$ gives us the number of $1 \le k \le x$ which are coprime to $d$, which is $\frac{x\phi(d)}{d} + O(2^{\omega(d)})$, see, e.g., here distribution of coprime integers . All other sums are $O(d^{1/2 + o(1)})$ by Polya--Vinogradov, so sum of sums is $\frac{x\phi(d)}{d2^{\omega(d)}} + O(d^{1/2+o(1)})$. If $d \le x^{2-o(1)}$ then the main term is bigger than twice the error term and so the sum is $x^{1-o(1)}$.
For non-squarefree $d$ we have to first of all cover the non-trivial characters modulo powers of $2$, and then the simplest approach for powers of $p$ is to just consider any number divisible by $p$ a non-residue (then we will have the sum of $3^{\omega(d)}$ terms, and the main term would be slightly smaller, but only by $d^{o(1)}$, so still miles bigger than the error term).
In fact, some google-fu led me to a blog post by Terence Tao https://terrytao.wordpress.com/2013/06/22/bounding-short-exponential-sums-on-smooth-moduli-via-weyl-differencing/ from which it appears that for smooth modules the bound for the character sums can be further improved, so likely this can give an even bigger lower bound for $d$ if we take your first point into account.