the $L^2$-norm of a signal is also applied as its energy!

Question

I am a newcomer in signal processing. I saw that the $L^2$-norm of a signal is also applied as its energy! How is this concept illustrated for those ones who are working in pure math.

Answer 1

Yes, the square of the $L_2$ norm of a signal is also by definition its energy $\mathcal{E}_x$.

The concept of signal energy :

$$ \mathcal{E}_x = \int_{-\infty}^{ \infty } x(t)^2 dt\tag{1} $$

is fundamentally based on the concept of energy (or work) in physics, as the Kinetic Energy of a particle with mass $m$ and velocity $v$ given by

$$ K = \frac{1}{2} m v^2 \tag{2}$$

There is also the concept of power defined as the time-rate of work $W(t)$.

$$ p(t) = \frac{dW(t)}{dt} \tag{3} $$

The relation between instantaneous power $p(t)$ and the energy is :

$$ \mathcal{E} = \int_{-\infty}^{\infty} p(t) dt \tag{4} $$

Electrical engineers ignore the mechanical roots, and rely on an electrical analog of energy as heat loss in an Ohmic resistor defined to be:

$$ \mathcal{E} = \int_{-\infty}^{\infty} p(t) dt \tag{5} $$

Where $p(t)$ is the instantaneous electric power associated with a current $i(t)$ passing through a linear time-invariant resistor $R$ , and is given by :

$$ p(t) = R \cdot i^2(t) \tag{6} $$

( $p(t) = v^2(t)/R $ is also an equivalent expression, based on Ohm's law $v(t) = R i(t)$)

Then the energy of the current, passing through a linear time-invariant system denoted by an Ohmic resistor $R$, is given by :

$$ \mathcal{E} = \int_{-\infty}^{\infty} R \cdot i^2(t) dt \tag{7}$$

Ignoring the resistor $R$ (or setting it to be $R=1$), and replacing the current variable $i(t)$ with a general unitless $x(t)$, we arrive at the mathematical definition of signal energy of as:

$$ \mathcal{E} = \int_{-\infty}^{\infty} x^2(t) dt \tag{8}$$

That being stated, in a parallel course, the study of normed linear Hilbert spaces also consider mathematical p-th Euclidean norm of a complex valued vector as :

$$ L_p = \left( \int_{-\infty}^{\infty} |x(t)|^p dt \right)^{1/p} \tag{9}$$

And you can see that the square of the case $p=2$ corresponds to the signal energy as defined in Eq.(8).

All of these can similarly be transferred to discrete-time domain.

Answer 2

From physics, energy is a term often used as a quantitative property. In other words, energy is a quantity that is preserved under some actions, transformations, etc. In signal processing (where physics vanish), this often takes the shape of a sum or an integral of a squared quantity for reals, or its modulus for complex data. We can write it symbolically for discrete or continuous time ($\cdot^H$ denotes the complex conjugate) by $\sum x[n]x^H[n]$ or $\int x(t)x^H(t)$. When they are well-defined (convergence, etc.), such quantities are mostly proportional to the square of some $L^2$ or $\ell^2$ norm. As said in other answers, energy and squared $L^2$ or $\ell^2$ norms are related by definition, they are at the center of complex Hilbert spaces.

Now, why are these concepts so important in signal processing? Because the linearity of systems is strongly linked to energy: minimizing an energy often results in linear equations, from simple averaging to generic convolution, with a special connection with Gaussian noises.

The crux of the squared norm use in DSP is related to orthogonality and unitarity: in signal and image processing, we pretend that some representations can preserve the energy (or up to a factor, or approximately), and be way more efficient for some processing methods: smoothing, adaptive filtering, separation, inversion, restoration, reconstruction, etc. Fourier, short-time Fourier, spectrograms, wavelets and other perform this energy conservation.

Lastly, energy preservation also plays a role in algorithmic stability.

Answer 3

How is this concept illustrated for those ones who are working in pure math.

I've never seen a pure mathematician need an illustration for a definition!

Really, the energy is defined as the sum of squares (discrete time) or the integral of squared (continuous time) signal.

At that point, it's not concept you have to apply, just a definition.

When leaving the math aspect of this and starting to care about the physicality:

This is compatible with the notion of power transported through a physical amplitude-changing phenomenon (like, say, a pressure wave in air, an electric voltage or a current on a wire, an electrical or magnetic field intensity, gravitational waves…): Instantaneous power is proportional to the square of amplitude, and energy is the integral of power over time.

Hence, that definition bridges the physical meaning of energy into signal procesing!

Answer 4

Possibly off-topic but in order to provide context, i.e. Parseval's identity:
I think a more general outlook should be pointed out. It's applicable in "reality" because we believe that Energy is conserved irrespective of description and there are equivalent similar relations for any of the linear transforms/representations; Laplace, Mellin, Fourier, Discrete, etc... The use of the L_2 norm is a reflection of this. Basically, they are weighted integrals/sums of coefficients/functions. Thus we need L_2 convergence/formulations to reach this conservation.
"More generally, Parseval's identity holds in any inner-product space, "
https://en.wikipedia.org/wiki/Parseval%27s_identity
A little sketchy and abstract but somewhat informative.