7

I have primarily dealt with Dense Matrices arising from Electrodynamics. However, I am interested in knowing where else Dense Matrices occur. I am especially interested in knowing where they occur in:

  1. Discretization of PDEs
  2. Engineering
  3. Science
  4. Statistics

Sparse matrices are present in FVM, FDM, FEM and so many others. Most of them have some or the other structure. Do "unstructured" dense matrices ever occur in problems?

I apologize for such a basic question but no one seems to know the answer to this (I tried Googling as well). I came across this but IMHO it isn't a satisfactory answer for my question.

Community
  • 1
Inquest
  • 3,394
  • 3
  • 26
  • 44

5 Answers5

9

To add to other responses,

  • Boundary integral methods for PDEs produce dense operators. These usually have structure allowing them to be applied using FMM or $\mathcal H$-matrix techniques, but direct summation is still commonly used. The boundary integral operators are usually compact perturbations of the identity, so the discrete operators are well approximated by low rank perturbations of the identity, consequently, Krylov methods tend to converge quickly.

  • Some diffuse scattering problems produce dense operators with little structure.

  • Inverse and optimization methods often produce dense operators. They are frequently Schur complements of sparse PDE operators and often have exploitable structure (e.g. well approximated by low rank perturbations of the identity).

  • Nonlinear time-periodic PDE problems (e.g. oceans; see also this for helicopters which is denser than other methods, but not completely dense).

Jed Brown
  • 25,650
  • 3
  • 72
  • 130
7

It is much more common for large sparse problems to be reduced to working with one or more smaller dense matrices. For example, the multifrontal method is designed to reorganize a factorization of a sparse matrix into partial factorizations of many smaller dense matrices.

Density functional theory is a case where the full matrices are often treated as unstructured and dense, as a nonlinear generalized eigenvalue problem needs to be solved and this is usually done through a sequence of linear dense generalized eigensolves.

Another instance is in saddle point problems arising in stochastic programming. And yet another is coupled cluster.

Jack Poulson
  • 7,599
  • 32
  • 40
  • I was aware of Multifrontal methods but then, they are dense conversions of sparse matrices. I am looking for problems where the matrix obtained is dense without any modifications. (Just like Tridiagonal for FDM of PDEs). Thanks for the Stochastic Programming lead. – Inquest Feb 01 '12 at 15:48
  • Note that there is no requirement that DFT or CC lead to dense matrices. DFT is already represented in sparse matrices for linear scaling purposes. Reduced-scaling CC methods are usually cast in terms of collections of small dense objects, i.e. block-sparse arrays. CC is often computed in terms of matrices but the mathematics is fundamentally tensorial. – Jeff Hammond Apr 13 '13 at 20:25
  • To be fair, I did use a significant number of weasel words. – Jack Poulson Apr 14 '13 at 02:17
5

In ODEs, dense matrices occur in Spectral Collocation Methods, e.g. as described in "Spectral Methods in Matlab" by L. N. Trefethen (SIAM, 2000) or as implemented in the Chebfun system.

Pedro
  • 9,573
  • 1
  • 36
  • 45
4

The classical choice of state variables used to represent the thermodynamic state in combustion simulations makes the Jacobian of the right hand side of the ODEs (or PDEs, if not using operator splitting) dense. Modern codes can get around this limitation by representing the thermodynamic state using an alternate choice of state variables (see On Upgrading the Numerics in Combustion Chemistry Codes for details).

Geoff Oxberry
  • 30,394
  • 9
  • 64
  • 127
4

The essential difference between dense and sparse linear systems is that dense systems express relationships in which each unknown depends on all (or many) of the other unknowns, while sparse systems express relationships in which each unknown depends on a small fixed number of other unknowns (often these are its "neighbors").

In the numerical discretization of PDEs by the method of lines, dense matrices occur whenever the basis functions of the spatial discretization have support over the whole problem domain (the support of a function is the region over which it takes non-zero values). The most common example is that of pseudospectral methods, as mentioned by Pedro. In that case, the basis functions are typically Fourier modes or Chebyshev polynomials.

As mentioned in the question, typical finite difference and finite element discretizations lead to sparse matrices, because they use basis functions with support equal to a few times the mesh spacing. However, there are finite difference and finite element discretizations that lead to dense matrices:

  • spectral collocation methods can naturally be viewed as finite difference methods where the finite difference stencil extends over the whole grid
  • Spectral Galerkin methods can naturally be viewed as finite element methods where the basis functions extend over the whole grid.
David Ketcheson
  • 16,522
  • 4
  • 54
  • 105
  • Note that these are at least somewhat "structured" in the sense that they can usually be applied using "fast" methods (e.g. FFT, but also more general representations that exploit effectively low-rank contributions from the far field). – Jed Brown Feb 01 '12 at 19:59
  • Hmmmm...I was answering the question title. I guess the body says something narrower. – David Ketcheson Feb 01 '12 at 20:33