15

I am using NonlinearModelFit for a thesis project.

I get quite different results if I change the Method to LevenbergMarquardt or QuasiNewton or ConjugateGradient and so on. My nonlinear model includes polynomials up to second order and a sigmoid function such as ArcTan or Tanh. I would like to know, which conditions Mathematica uses to choose the best algorithm if I set the Method as Automatic. It works sometimes, but most of the time it doesn't and I do not have the knowledge to say which method should be the best in my case. Just by trying out, I found Gradient and sometimes ConjugateGradient to work the best for my purpose.

I also cannot find anywhere on the Internet which exact implementation of these methods is used inside Mathematica and how does it change the results. Here maybe an interesting link: Optimization Algorithms

Thanks!

J. M.'s missing motivation
  • 124,525
  • 11
  • 401
  • 574
Santiago
  • 1,201
  • 8
  • 16
  • If necessary I can post some data and some output from Mathematica. – Santiago Mar 22 '13 at 10:15
  • Have you read this http://reference.wolfram.com/mathematica/tutorial/UnconstrainedOptimizationOverview.html? – Dr. belisarius Mar 22 '13 at 10:27
  • No, I hadn't. Thanks for the link! Probably this will explain some things, but still I think there is nowhere an explanation in the context of Nonlinear Model Fit. – Santiago Mar 22 '13 at 14:13
  • 2
    To the contrary -- this tutorial (note there is also a pdf version you can download and read at your convenience) describes all of the optimization technologies -- of which NonlinearModeFit is just one. http://www.wolfram.com/learningcenter/tutorialcollection/UnconstrainedOptimization/UnconstrainedOptimization.pdf – bill s Apr 27 '13 at 06:33
  • 1
    NonlinearModelFit seems to be implemented as a wrapper around FindFit, but the latter is kernel code and not accessible for inspection. I would suggest contacting WRI support to ask them which method Automatic translates to and under what conditions. – Oleksandr R. May 02 '13 at 17:15

1 Answers1

18

Unfortunately the Method option is not documented in detail on the NonlinearModelFit documentation page. To summarize what we know so far (comments, documentation, etc.):

NonlinearModelFit can either use numerical local optimization, or numerical global optimization.

  • Local optimization is the same as used by FindMinimum and related functions. The possible method options are documented in detail here. NonlinearModelFit will take these options directly. Example:

    NonlinearModelFit[..., Method -> {"Newton", "StepControl" -> "TrustRegion"}]

  • Global optimization is the same as used by NMinimize and related functions. The available method options are detailed here. The syntax to use is 

    NonlinearModelFit[..., Method -> {NMinimize, Method -> ...}]

Further documentation on the constrained optimization is here.

Szabolcs
  • 234,956
  • 30
  • 623
  • 1,263