5

I have to minimize a function where the evaluation of one parameter set takes very long (around 5sec) and discovered alongside, that NMinimize seems to call this function multiple times with the exact same values. As an example, consider the code

f[x_?NumericQ] := (Print[N[x, 10]]; x^2);
NMinimize[{f[x]}, x, Method -> "RandomSearch"]

The first couple of lines of the output are:

-0.829053
-0.829053
-0.829053
-0.829052
-0.329053
-0.329053
-0.329052
-8.85837*10^-9
-8.85837*10^-9
 6.0428*10^-9

Am I missing something? The same output is also produced by setting PrecisionGoal -> 3 and AccuracyGoal -> 3, thus we can rule out that the numbers in the output are just identical representations of different numbers.

Frederik Ziebell
  • 1,053
  • 8
  • 21
  • 1
    You can do this with g[x_] = x^2, {sol, data} = Reap[NMinimize[{g[x]}, x, Method -> "RandomSearch", EvaluationMonitor :> Sow[{x, g[x]}]]]; and look at data. – b.gates.you.know.what Feb 13 '14 at 10:18
  • 3
    I've observed this as well. I haven't investigated it thoroughly (you could use the diagnostic output if you want to see in detail what's going on) but the problem seems to be that in most cases Mathematica doesn't save the objective function value anywhere, so wherever it's required, it just gets reevaluated. NMinimize is great because of its very general constraint handling, but it isn't the most efficient choice in most cases. Perhaps try FindMinimum or, if you don't need constraints, my own Nelder-Mead optimizer. – Oleksandr R. Feb 13 '14 at 11:12
  • Hmm, I wonder why manual memonization does not help. ClearAll@f; f[x_?NumericQ] := (f[x] = (Print[N[x, 10]]; x^2)); NMinimize[f[x], x, Method -> "RandomSearch"]. So I guess f gets converted/compiled under the hood... – Ajasja Feb 13 '14 at 14:42
  • 2
    In addition to the useful suggestions from @Oleksandr R., you could define your function so that it memoizes, that is f[x_?NumericQ]:= f[x]=.... This will not prevent all reevaluations but I find that it can help. Caveat: you might need to sporadically clear out definitions so as not to use too much memory. – Daniel Lichtblau Feb 13 '14 at 14:50
  • I find the memoization works: ClearAll[f, g]; g = {}; f[x_?NumericQ] := f[x] = (g = {g, x}; x^2); NMinimize[f[x], x, Method -> "RandomSearch"]; DuplicateFreeQ[Flatten[g]] returns True. – ecoxlinux Feb 13 '14 at 15:45
  • @ecoxlinux, no it does not, because ClearAll[f, g]; g = {}; f[x_?NumericQ] := (g = {g, x}; x^2); NMinimize[f[x], x, Method -> "RandomSearch"]; DuplicateFreeQ[Flatten[g]] returns False. In your example, if f[x] is requested the second time for the same x-value, g={g,x} is not executed. – Frederik Ziebell Feb 14 '14 at 08:00
  • @FrederikZiebell I don't understand your comment. I stated that the above code seems to always returns True in my computer. Are you saying that it returns False in yours? – ecoxlinux Feb 14 '14 at 15:23
  • I have doubts about the usefulness of memoization in this context. Most of the memoized values will never be accessed again, so unless you do it in such a way as to save only a few of the most recent results, it represents mostly wasted effort as a workaround for a design issue in NMinimize. – Oleksandr R. Feb 15 '14 at 17:18

0 Answers0