I am trying to integrate this equation:
$$ \frac{dz}{dt} = \alpha \cdot (s(t)-z(t-1)) $$
I came up with this equation to model a "leaky integrator" system where s(t) are some samples from the environment and z(t) is a "decision variable". Thus this system can continuously decide between 2 alternatives based on the sign of z(t).
However, I'm not sure how to integrate the equation and solve for z(t), mainly because of the term z(t-1) which Wolfram Alpha seems to struggle with... Any tips?
Your differential equation is a delay differential equation (DDE). You can find a quick overview on this here.
When solving a homogeneous solution of an linear DDE you could reduce it to it's characteristic equation using the substitution $z\left( t \right) := e^{\lambda \cdot t} \implies z^{\left( n \right)}\left( t + \tau \right) \equiv \lambda^{n} \cdot e^{\lambda \cdot \left( t + \tau \right)}$. You will get no polynomial like in ODEs but a similar equation wich you can solve using the Lambert-W Function $\operatorname{W}_{k}$:
\begin{align*} \frac{\operatorname{d}z\left( t \right)}{\operatorname{d}t} &= \alpha \cdot \left( s\left( t \right) - z\left( t - 1 \right) \right)\\ \frac{\operatorname{d}z_{h}\left( t \right)}{\operatorname{d}t} &= -\alpha \cdot z_{h}\left( t - 1 \right) \tag{$z_{h}\left( t \right) := e^{\lambda \cdot t}$}\\ \lambda \cdot e^{\lambda \cdot t} &= -\alpha \cdot e^{\lambda \cdot \left( t - 1 \right)}\\ \lambda \cdot e^{\lambda \cdot t} &= -\alpha \cdot e^{\lambda \cdot t} \cdot e^{-\lambda} \tag{$\div e^{\lambda \cdot t}$}\\ \lambda &= -\alpha \cdot e^{-\lambda} \tag{$\cdot e^{\lambda}$}\\ \lambda \cdot e^{\lambda} &= -\alpha \tag{$\operatorname{W}_{k}\left( \cdot \right)$}\\ \lambda &= \operatorname{W}_{k}\left( -\alpha \right)\\ \end{align*} where $k$ is any integer ($k \in \mathbb{Z}$).
So $$\fbox{$ \begin{align*} z_{h}\left( t \right) &= \text{c} \cdot e^{\operatorname{W}_{k}\left( -\alpha \right) \cdot t}\\ \end{align*} $}$$ or $$\fbox{$ \begin{align*} z_{h}\left( t \right) &= \sum\limits_{k = -\infty}^{\infty}\left[ \text{c}_{k} \cdot e^{\operatorname{W}_{k}\left( -\alpha \right) \cdot t} \right]\\ \end{align*} $}$$ where $\left\{ \text{c},\, \text{c}_{k} \right\} \in \mathbb{C}$ are arbitrary complex constants.
You can use the Laplace transform $\mathcal{L}$ and its inverse to find the particular solution here. The special properties (here Frequency-domain derivative (I'll shorten it with Fdd) and Frequency shifting (I'll shorten it with Fs) allow for simple simplifications. Let's say $\tilde{z}$ is the inverse Laplace transform of $z$ the we could solve:
\begin{align*} \frac{\operatorname{d}z\left( t \right)}{\operatorname{d}t} &= \alpha \cdot \left( s\left( t \right) - z\left( t - 1 \right) \right)\\ \frac{\operatorname{d}z\left( t \right)}{\operatorname{d}t} &= \alpha \cdot s\left( t \right) - \alpha \cdot z\left( t - 1 \right) \tag{$\cdot \left( -1 \right)$}\\ -\frac{\operatorname{d}z\left( t \right)}{\operatorname{d}t} &= \alpha \cdot z\left( t - 1 \right) - \alpha \cdot s\left( t \right) \tag{$\mathcal{L}_{t}^{-1}\left[ \cdot \right]\left\{ x \right\}$}\\ \mathcal{L}_{t}^{-1}\left[ -\frac{\operatorname{d}z\left( t \right)}{\operatorname{d}t} \right]\left\{ x \right\} &= \mathcal{L}_{t}^{-1}\left[ \alpha \cdot z\left( t - 1 \right) - \alpha \cdot s\left( t \right) \right]\left\{ x \right\} \tag{Fdd}\\ x \cdot \tilde{z}\left( x \right) &= \mathcal{L}_{t}^{-1}\left[ \alpha \cdot z\left( t - 1 \right) - \alpha \cdot s\left( t \right) \right]\left\{ x \right\} \tag{linearity}\\ x \cdot \tilde{z}\left( x \right) &= \alpha \cdot \mathcal{L}_{t}^{-1}\left[ x\left( t - 1 \right) \right]\left\{ x \right\} - \alpha \cdot \mathcal{L}_{t}^{-1}\left[ s\left( t \right) \right]\left\{ x \right\} \tag{Fs}\\ x \cdot \tilde{z}\left( x \right) &= \alpha \cdot e^{x} \cdot \tilde{z}\left( x \right) - \alpha \cdot \mathcal{L}_{t}^{-1}\left[ s\left( t \right) \right]\left\{ x \right\}\\ \end{align*}
Now we got a linear equation, which can be easily solved:
\begin{align*} x \cdot \tilde{z}\left( x \right) &= \alpha \cdot e^{x} \cdot \tilde{z}\left( x \right) - \alpha \cdot \mathcal{L}_{t}^{-1}\left[ s\left( x \right) \right]\left\{ x \right\} \tag{$-\left( \alpha \cdot e^{x} \cdot \tilde{z}\left( x \right) \right)$}\\ \left( x - \alpha \cdot e^{x} \right) \cdot \tilde{z}\left( x \right) &= -\alpha \cdot \mathcal{L}_{t}^{-1}\left[ s\left( t \right) \right]\left\{ x \right\} \tag{$\cdot \left( -1 \right)$}\\ \left( \alpha \cdot e^{x} - x \right) \cdot \tilde{z}\left( x \right) &= \alpha \cdot \mathcal{L}_{t}^{-1}\left[ s\left( t \right) \right]\left\{ x \right\} \tag{$\div \left( \alpha \cdot e^{x} - x \right)$}\\ \tilde{z}\left( x \right) &= \frac{\alpha \cdot \mathcal{L}_{t}^{-1}\left[ s\left( t \right) \right]\left\{ x \right\}}{\alpha \cdot e^{x} - x} \tag{$\mathcal{L}_{x}\left[ \cdot \right]\left\{ t \right\}$}\\ z\left( t \right) &= \mathcal{L}_{x}\left[ \frac{\alpha \cdot \mathcal{L}_{t}^{-1}\left[ s\left( t \right) \right]\left\{ x \right\}}{\alpha \cdot e^{x} - x} \right]\left\{ t \right\}\\ \end{align*}
We get: $$\fbox{$ \begin{align*} z_{p}\left( t \right) &= \mathcal{L}_{x}\left[ \frac{\alpha \cdot \mathcal{L}_{t}^{-1}\left[ s\left( t \right) \right]\left\{ x \right\}}{\alpha \cdot e^{x} - x} \right]\left\{ t \right\}\\ \end{align*} $}$$
This is the particular solution.
Note that you could also use at first the Laplace transform and then it's inverse. You would get a solution like Robert Israel.
We know $z\left( t \right) = z_{p}\left( t \right) + z_{h}\left( t \right)$, so we would get: $$\fbox{$\fbox{$ \begin{align*} z\left( t \right) &= \mathcal{L}_{x}\left[ \frac{\alpha \cdot \mathcal{L}_{t}^{-1}\left[ s\left( t \right) \right]\left\{ x \right\}}{\alpha \cdot e^{x} - x} \right]\left\{ t \right\} + \sum\limits_{k = -\infty}^{\infty}\left[ \text{c}_{k} \cdot e^{\operatorname{W}_{k}\left( -\alpha \right) \cdot t} \right]\\ \end{align*} $}$}$$
This is known as a delay-differential equation. Note that for an "initial condition", you want to specify $z(t)$ on an interval of length $1$, say $(-1,0)$, rather than at just one point.
One way to try solving it is with Laplace transform. Since you're using $s$ in your equation, let me use $u$ as the Laplace transform variable. Then, with $Z(u)$ and $S(u)$ the Laplace transforms of $z$ and $s$, the Laplace transform of your equation is
$$u Z(u) - z(0) = \alpha S(u) - \alpha \int_{-1}^0 z(t) e^{-u(t+1)} \; dt - e^{-u} Z(u)$$
so
$$Z(u) = (u + e^{-u})^{-1} \left(z(0) + \alpha S(u) - \alpha \int_{-1}^0 z(t) e^{-u(t+1)}\; dt \right)$$
Supposing that $S(t) = \sum_{k=0}^{n_s}a_k t^k$ and $z(t)= \sum_{k=0}^{n_z}b_k t^k$ this last for $t\in [-1,0]$, after the Laplace transform we have
$$ s\hat z(s)-\alpha \hat S(s)+\alpha e^{-s}\hat z(s) -\int_{-1}^0z(t)e^{-s(t+1)}dt = z_0 $$
Regarding $\int_{-1}^0z(t)e^{-s(t+1)}dt$ we know that $\int_{-1}^0t^k e^{-s(t+1)}dt = T_k(s)$ which obeys the recurrence
$$ T_{k+1}(s)-\frac ks T_k(s) = \frac{(-1)^k}{s},\ \ \ \ T_0(s) = \frac{1-e^{-s}}{s} $$
so we can write
$$ \hat z(s) = \frac{z_0+\alpha \hat S(s)+T(s)}{s+\alpha e^{-s}}=\frac{z_0+\alpha \hat S(s)+T(s)}{s(1+\frac{\alpha e^{-s}}{s})} $$
with $T(s) = \sum b_k T_k(s)$ and now assuming $\|\frac{\alpha e^{-s}}{s}\| < 1$ we have
$$ \frac{1}{1+\frac{\alpha e^{-s}}{s}}=\sum_{k=0}^{\infty}(-1)^k\left(\frac{\alpha e^{-s}}{s}\right)^k $$
making $\hat z(s)$ invertible.
