The $\mathcal{Z}$-transform of a discrete-signal is namely $$F(z) = \sum_{n=-\infty}^{\infty} f[k] z^{-n}$$ and so if I have a signal in the $\mathcal{Z}$-plane:
$$G(z) = z-1$$
I would be having a time-domain signal like so: $$g[k] = \delta[k+1] - \delta[k]$$
So my question is, can I say that this signal is finite? For example, if I use $k = 1$, $g[k]$ would return $\delta[1]-\delta[0]$?
I am a little confused in this case and as to how we can interpret $z-1$ correctly in time domain.
More often than not on a practical level, such $G(z)$ as given by the OP refers to the $\mathcal{Z}$-transform of the unit sample response of a linear time invariant system. The discrete-signal is the response the system would provide given a unit sample ($\delta[k]$) at the input to the system.
In this case we would have a non-causal system, since the output is determined from the present input and an input one sample in the future. Transfer functions in the $\mathcal{Z}$ domain are typically a ratio of polynomials (here in this case we have $G(z)=z+1$ which is a numerator polynomial of $z+1$ and the denominator is $1$). Any system where the numerator is of higher power than the denominator (such as $(z+1)/1$) would be non-causal as described below.
It is common to rewrite the transfer function using negative powers of $z$, since conveniently $z^{-1}$ is the transfer function of a unit sample delay and therefore the transfer function will lead directly to an implementation in terms of combining delays, adds, and multiplies. So $z$, or the inverse of $z^{-1}$ is a unit sample lead.
That said, $G(z)$ in this case can be written as:
$$G(z) = z-1 = \frac{1-z^{-1}}{z^{-1}}$$
With the numerator as describing the system that is the past sample subtracted from the current sample, and the denominator serving to shift the whole response forward in time by one sample (and therefore non-causal).
If we had the numerator alone (as $G(z) = 1-z^{-1}$ or equivalently $G(z)= \frac{z-1}{z}$ we would have the following system implementation, where here we see how this falls right out of the transfer function as described with negative powers of $z$:
So we have in this case where it is causal:
$$Y(z) = X(z)-X(z)z^{-1}$$
Where $X(z)$ represents the $\mathcal{Z}$-transform of the input signal and $Y(z)$ represents the $\mathcal{Z}$-transform of the output signal.
So the transfer function from input to output in this case would be:
$$\frac{Y(z)}{X(z)} = 1-z^{-1}$$
Since $z^{-1}$ is the $\mathcal{Z}$-transform of a unit sample delay, we'll see this show up in block diagrams where any unit sample delay will be labeled with $z^{-1}$, and similarly an $N$ sample delay of any number of samples would be $z^{-N}$.
On the other hand, the system given by $z-1$ is not realizable, since it is not causal. It would require us to use a device that doesn't delay the input by one sample, but instead will advance the input in time by one sample: meaning produce the input we are going to apply one sample before we apply it! If we were to cascade such a system with a unit sample delay (multiply by $z^{-1}$; when we cascade systems in series, we multiply the transfer functions), all would be well again and the result would be a causal implementation resulting in $1-z^{-1}$.
