What is lost in terms of approximation, when writing a problem in terms of a Differential Algebraic Equation (DAE) system rather than an ODE system?

Question

It may be that we have a model where the following equation holds for some phenomenon:

$$(1)\quad x + y + z = T$$

Importantly, $T$ is a constant, i.e.: $$(2) \quad \frac{\mathrm{d}T(t)}{\mathrm{d}t} = 0$$

We may be interested in the evolution of $x$, $y$ and $z$ as governed by their differential equations. If we want to make sure that we maintain condition $(2)$ (not guaranteed for any numerical integration scheme), then we may represent the problem as follows:

$$ (3a)\quad \frac{\mathrm{d}x}{\mathrm{d}t} = f(x, y, z, t)$$

$$ (3b)\quad \frac{\mathrm{d}y}{\mathrm{d}t} = g(x, y, z, t)$$

$$ (3c)\quad \frac{\mathrm{d}z}{\mathrm{d}t} = \frac{\mathrm{d}T(t)}{\mathrm{d}t}- f(x,y,z,t) - g(x, y, z, t) = - f(x,y,z,t) - g(x, y, z, t)$$

So, some program might calculate the evolution of $x$, $y$ and $z$ by numerical integrating $x$ and $y$, while using condition $(2)$ to determine $z$ (generally, abd during the calculations for $x$ and $y$).

One takeaway I have had from a numerical analysis course I was a part of recently is that "if you're not violating some conservation rule in the phenomenon being modelled, then you have the exact solution" (and numerical solutions are not exact solutions, usually). Thus, even though I might feel nice about saving $(2)$, I know I am losing something somewhere else -- what is it that I am losing?

An idea I have have is that $(3)$ essentially allows a numerical integration scheme to assume that any difference in $T$ is solely made up by $z$, rather than by some combination of $x$, $y$ and $z$? Or, put differently, any error in the numerical integration scheme in violating $(2)$ is chalked up to $z$, so we may not be getting the actual dynamics of $z$, but rather, the dynamics of $z$ along with some error based on how $(2)$ is being violated by the numerical scheme?

How might one more precisely explain what the downside of using a DAE system is in terms of "what is lost in approximation"?

Answer 1

Here's a start to answering your question:

One takeaway I have had from a numerical analysis course I was a part of recently is that "if you're not violating some conservation rule in the phenomenon being modelled, then you have the exact solution" (and numerical solutions are not exact solutions, usually).

Kirill and JedBrown pointed out situations where conservation doesn't hold; open systems in thermodynamics are another situation. Even in those cases, you don't have the exact solution. Here's a stupidly trivial example that can't be solved exactly on a computer:

\begin{align} \dot{x} = 0, \quad x(0) = 1/3 \end{align}

One-third is not exactly representable in floating-point arithmetic because it can't be expressed as a finite decimal in base-2. So generally speaking, you should expect any numerical solution to have some amount of error in it (even if that error is small); in this case, the error is going to be in the machine representation of one-third.

An idea I have have is that (3) essentially allows a numerical integration scheme to assume that any difference in $T$ is solely made up by $z$, rather than by some combination of $x$, $y$ and $z$? Or, put differently, any error in the numerical integration scheme in violating (2) is chalked up to $z$, so we may not be getting the actual dynamics of $z$, but rather, the dynamics of $z$ along with some error based on how (2) is being violated by the numerical scheme?

I don't see why this situation would be true in general, and numerical schemes normally have tolerances so that you could control the error in the numerical solution. In many cases, you can control the error on a per-variable basis, so conceivably, you could place tight tolerances on $y$ and $z$, plus a loose tolerance on $x$, and expect that you might have more error in $x$ than in $y$ or $z$.

How might one more precisely explain what the downside of using a DAE system is in terms of "what is lost in approximation"?

As you pointed out, you can express some physical systems in multiple ways. Any ODE can be expressed as a DAE; you can also take advantage of conservation laws to rewrite your system of differential equations. In general, rewriting an ODE as a DAE (in a trivial way, re-expressing $\dot{y} = f(y)$ as $\dot{y} - f(y) = 0$) and using numerical methods is more expensive, but not always. For instance, if when using sparse direct methods to solve the linear systems for the numerical methods, the Jacobian matrix of the DAE is much more sparse than the Jacobian matrix of the right-hand-side of the ODE, then solving the DAE may be faster. For a while, this approach was advocated in combustion.

Some ODE methods don't preserve certain types of invariants (e.g., the conservation laws in your example), so the numerical solution might be mostly accurate, but fail to satisfy the invariant in a noticeable way. An example would be something like using a Runge-Kutta method for some Hamiltonian system like orbital dynamics; over short times, your numerical solution will be fine, but over long times, failure to conserve energy will start to introduce significant error in the numerical solution. This shortcoming can be corrected by using symplectic methods (e.g., Verlet), or you could solve the DAE, assuming the DAE is stable.