Consider a (test) charge located at $\vec{r}(t)$ in a static electric field with potential $V(\vec{r})$. The energy for this system is given by
$$E = \frac{1}{2} m \dot{\vec{r}}(t)^2 + q V(\vec{r}(t)).$$
(One could add in a magnetic field without altering this energy.) It is well known that the potential can be changed by a constant without altering the physics; we can fix this by choosing the potential at $\infty$ to be zero (let us assume that all fields decay as $r \to \infty$ for simplicity).
Under a gauge transformation, we have $$ V \to V - \frac{\partial f}{\partial t}, \qquad \vec{A} \to \vec{A} + \nabla f. $$ If I choose $f$ to be a function of time only, this is equivalent to modifying the potential at $\infty$ in a time-dependent manner, which essentially adds a time-variable offset to the energy expression above. If I choose $f$ to be a function of position only, then the electric potential is unchanged, and the expression for the energy remains unchanged.
My question is how to interpret the case where $f$ depends on both space and time? Under such a gauge transformation, the energy of the system becomes $$ E = \frac{1}{2} m \dot{\vec{r}}(t)^2 + q V(\vec{r}(t)) - q \frac{\partial f}{\partial t}\bigg|_{\vec{r}(t)}. $$ What is the interpretation of this term? Why is the energy not well-defined, up to a choice of reference point? While I'm ignoring the field created by the charge, I don't think I need to consider it explicitly to understand this. Certainly in the test charge limit, it should have no effect. Note that the energy in the background field is invariant under the gauge transformation, as it depends on integrals of E and B only. What am I missing?
Edit: I'm well aware that the Hamiltonian for such a system is not gauge invariant, though the equations of motion are. I'm trying to understand why energy is not a "good" quantity to describe this situation.