7

Are there any drawbacks to using the Method of Manufactured Solutions for convergence testing and verification studies? I really can't think of any.

user27504
  • 321
  • 1
  • 5

1 Answers1

9

I think MMS is generally a good technique.

If I were to offer criticism, however, it would be that in my discipline (electromagnetic radiation and scattering), it leads one to spending time/money on implementing mathematics that really don't correspond to actual use cases. MMS requires injecting arbitrary dirichlet/neumann/robin data on boundaries and arbitrary volume sources, but real physical models don't actually use these (they use port boundary conditions, modal expansion boundary conditions, total-field/scattered-field decompositions for sources.. all of these are weirder constructions that fall outside the scope of basic MMS). Overall this makes MMS feel rather tautological: it insists of me that I write extra code that is only of use/coveraged by MMS itself, and yet MMS remains rather silent when it comes to bigger/harder questions.

As an alternative to implementing MMS for electromagnetism, I recommend instead to implement cavity eigenproblem tests (ie compute resonant frequencies of a closed cavity of canonical shape, subject to zero-valued dirichlet or neumann conditions). It can tell you similar things as MMS: am I obtaining the correct rates of convergence as I fiddle with geometry/basis orders and mesh density. Unlike MMS, this doesn't require any additional nonzero boundary-data/body-load terms. And yet, it actually solves a physical problem of genuine engineering interest! (resonant frequencies of general shapes). It does carry the drawback of more complicated algebraic machinery (solving a sparse eigenproblem instead of sparse linear system), but there are good black-box libraries for either case.

All that said, my opinion is shaped by my domain of interest (linear electromagnetism), so other PDE's might not have such a tidy/useful alternative to MMS. I certainly won't criticize other practitioners for thorough testing! It's more a sentiment that, in my use cases, MMS entails additional work but doesn't give very much additional information.

rchilton1980
  • 4,862
  • 13
  • 22
  • A very nice answer. That also feels true for Fluid Mechanics sometimes. I mean, for many flow situations you don't need space/time dependent boundary conditions and source term, yet if you want to do a thorough MMS you need to do this generalization in your code... – BlaB May 10 '18 at 17:14
  • @BlaB But isn't the purpose of MMS to verify all parts of the code, leaving no/less possibility of code/numerical error? I understand what you mean by the unnecessary flow situations as it relates to actual scenarios but, in the context of MMS, isn't that irrelevant? – user27504 May 10 '18 at 18:56
  • You are right. My feeling is that MMS is a very good technique for the verification of a CFD code (even if it becomes more complex when you have turbulence / wall functions). However, sometimes it requires additional efforts which are orthogonal to your main code development and this may hinder your implementation of MMS. Not everyone wants to spend an extra 1-2 weeks to allow for time and space dependent source term / boundary conditions that can be derived from equations, etc. MMS should indeed verify all parts of your code, but for that, all parts of your code must be MMS compatible – BlaB May 10 '18 at 19:11
  • Ah I see what you're saying. I haven't experienced that yet. I can see how that would be a PITA for CFD when you start adding in turbulence terms and other stuff. – user27504 May 10 '18 at 19:17
  • You raise a valid critique of MMS - but I think the conclusion to draw is that it is one of many validation techniques which should be employed in order to build confidence in a solver.

    MMS is weak at exercising real-world scenarios, but strong at verifying mathematical properties of a solver. This is important because a test shouldn't verify too many properties simultaneously so that when it fails we know why it failed.

    All solvers are built from some theoretical/mathematical idea of what should work on the given physics, MMS verifies that those assumptions are correct.

     – Reid.Atcheson May 10 '18 at 20:15
  • @Reid Acheson is correct to advise applying a variety of tests. MMS produces a solution to an equation with source terms. The correct handling of source terms is a contentious issue, and their presence can create spontaneous motion in some cases. My feeling is that MMS raises as many issues as it solves. – Philip Roe May 10 '18 at 22:56