This is impossible. We deal with the Noetherian case first, and then deduce the general case.
Lemma. Let $R$ be a reduced infinite Noetherian ring without idempotents, and let $\mathfrak p \subseteq R$ be a prime. If $R/\mathfrak p$ is finite, then there exists a prime $\mathfrak q \subsetneq \mathfrak p$. For every such $\mathfrak q$, the ring $R/\mathfrak q$ is infinite.
Proof. Since $R/\mathfrak p$ is a finite domain, it is a finite field, hence $\mathfrak p$ is maximal. If there does not exist such $\mathfrak q$, then $\mathfrak p$ is minimal as well. This means that the constructible set $V(\mathfrak p) = \{\mathfrak p\}$ is stable under generisation and specialisation, hence it is a clopen subset of $\operatorname{Spec}(R)$; see e.g. [Tag 0542]. Since $R$ has no nontrivial idempotents, $\operatorname{Spec} R$ is connected, so this forces $\mathfrak p$ to be the unique prime ideal. But since $R$ is reduced, we conclude that $\mathfrak p = (0)$, so $R$ itself is finite, contradiction.
Hence, $\mathfrak p$ cannot be minimal. Thus, there exists $\mathfrak q \subsetneq \mathfrak p$. But $R/\mathfrak q$ cannot be finite, because a finite domain is a field, which would say that $\mathfrak q$ is maximal. $\square$
Remark. We used the Noetherian hypothesis in the statement that $V(\mathfrak p)$ is constructible. In a general ring, there is a retrocompactness assumption on the complement $D(\mathfrak p)$, which is satisfied for example if $\mathfrak p$ is (the radical of) a finitely generated ideal. See [Tag 04ZC] for details.
There is a more elementary argument if you don't want to use constructible sets. Indeed, assume $R$ has minimal primes $\mathfrak p_i$ different from $\mathfrak p$. Since $R$ is Noetherian, there are finitely many such, and since $\operatorname{Spec}(R)$ is connected, there must be an $i$ such that $V(\mathfrak p) \cap V(\mathfrak p_i) \neq \varnothing$. But that means that $\mathfrak p \in V(\mathfrak p_i)$, i.e. $\mathfrak p_i \subseteq \mathfrak p$, contradicting minimality of $\mathfrak p$.
Lemma. Let $R$ be a reduced infinite Noetherian ring without idempotents, and let $f \in R[x]$. If $f(r) = 0$ for all $r \in R$, then $f = 0$.
Proof. Let $\mathfrak p$ be a prime for which $R/\mathfrak p$ is infinite. Denote by $\bar{r}$ the reduction of $r$ in $R/\mathfrak p$. We clearly have $\bar f(\bar r) = \overline{f(r)}$ for all $r \in R$, so we conclude that $\bar f \in (R/\mathfrak p)[x]$ vanishes at all elements of $R/\mathfrak p$. Since $R/\mathfrak p$ is an infinite domain, this forces $\bar f = 0$, i.e. $f \in \mathfrak pR[x]$. Hence,
$$f \in \bigcap_{\substack{\mathfrak p\\ |R/\mathfrak p| = \infty}} \mathfrak p R[x].\label{Eq 1}\tag{1}$$
But by the lemma above, every prime for which $R/\mathfrak p$ is finite contains a prime $\mathfrak q$ for which $R/\mathfrak q$ is infinite, so the intersection in (\ref{Eq 1}) is equal to
$$\bigcap_{\mathfrak p} \mathfrak p R[x].$$
That is, every coefficient of $f$ is in $\bigcap_{\mathfrak p} \mathfrak p = \mathfrak{nil}(R)$, which is $0$ by assumption. $\square$
Corollary. Let $R$ be any reduced infinite ring without idempotents, and let $f \in R[x]$. If $f(r) = 0$ for all $r \in R$, then $f = 0$.
Proof. If $R$ is a field, then it is an infinite field, so we are done. Otherwise, let $r_0 \in R\setminus\{0\}$ be an element that does not have an inverse, and let $R'\subseteq R$ be a finitely generated subring containing $r_0$ such that $f \in R'[x]$. For example, we can take the subring generated by the coefficients of $f$ and $r_0$.
Then $R'$ is reduced and has no nontrivial idempotents, and $R'$ is Noetherian since it is finitely generated. If $R'$ is finite, then it is Artinian, hence local since it has no idempotents, hence a field since it has no nilpotents. This contradicts the assumption that $r_0$ does not have an inverse.
Thus, $R'$ is infinite, and $f(r') = 0$ for all $r' \in R'$. Therefore, $f = 0$ by the lemma above applied to the ring $R'$. $\square$
