Wagner's trick to monitor FindRoot no longer works in Mathematica version 8

Question

I recently learned that we can use Print as a monitoring tool. My favorite is Ted Ersek's example

h[f1[a1], f2[e][a2]] /. (a_ /; Print[a] :> 0)

which reveals the order in which ReplaceAll goes through parts of an expression. Clearly this is a very useful trick.

On page 41 of his book, David Wagner gave the example

FindRoot[Print[x]; Sin[x] - Cos[x], {x, .5}]

which prints the sequence of iterates generated by FindRoot along the way. That code no longer works on Mathematica version 8:

In[22]:= FindRoot[Print[x]; Sin[x] - Cos[x], {x, .5}]
During evaluation of In[22]:= x
Out[22]= {x -> 0.785398}

At first I thought this is related to the fact that we now have the EvaluationMonitor option, and FindRoot somehow weeds out superfluous subexpressions like Print. Now I know this is probably false because

Plot[Print[x]; Sin[x], {x, Pi/4, Pi/2}]

still lists all the points considered during its evaluation.

Why does FindRoot ignore Print?

I've learned recently that Plot, probably as well as FindRoot don't necessarily evaluate the whole first argument, as is, in every iteration. They sometimes partially evaluate first, and I am not sure in what cases. So you probably simply shouldn't put there what you want evaluated every time. Probably since EvaluationMonitor exists they have taken the liberty to put some optimizations in the expression preprocessing, where they felt like it. — Rojo, Jun 09 '12 at 03:00

Alexey Popkov · Accepted Answer · 2012-06-12T13:57:12.037

26

Try the Evaluated -> False option:

FindRoot[Print[x]; Sin[x] - Cos[x], {x, .5}, Evaluated -> False]

During evaluation of In[3]:= 0.5

During evaluation of In[3]:= 0.5

During evaluation of In[3]:= 0.5

During evaluation of In[3]:= 0.793408

During evaluation of In[3]:= 0.793408

During evaluation of In[3]:= 0.793408

During evaluation of In[3]:= 0.785398

During evaluation of In[3]:= 0.785398

During evaluation of In[3]:= 0.785398

During evaluation of In[3]:= 0.785398

During evaluation of In[3]:= 0.785398

During evaluation of In[3]:= 0.785398

Out[3]= {x -> 0.785398}

As to the Plot, try

Plot[Print[x]; Sin[x], {x, Pi/4, Pi/2}, Evaluated -> True];

During evaluation of In[5]:= x

As you see, the behavior is exactly the same as it is for FindRoot by default. The difference in default behavior can be explained by default values of the Evaluated option:

Options[#, Evaluated] & /@ {Plot, FindRoot}

{{Evaluated -> Automatic}, {Evaluated -> True}}

It seems that the Automatic value is equivalent to False in this case.

Another approach

As Albert Retey mentioned in the comment, the "standard" (and documented) way to monitor evaluations is to define objective function as black-box function by restricting its argument to numerical values only:

In[1]:= f[x_?NumericQ] := (Print[x]; Sin[x] - Cos[x])
FindRoot[f[x], {x, .5}]

During evaluation of In[1]:= 0.5

During evaluation of In[1]:= 0.5

During evaluation of In[1]:= 0.5

During evaluation of In[1]:= 0.793408

During evaluation of In[1]:= 0.793408

During evaluation of In[1]:= 0.793408

During evaluation of In[1]:= 0.785398

During evaluation of In[1]:= 0.785398

During evaluation of In[1]:= 0.785398

During evaluation of In[1]:= 0.785398

During evaluation of In[1]:= 0.785398

During evaluation of In[1]:= 0.785398

Out[2]= {x -> 0.785398}

Additional comparisons

Here I switch off autocompilation for not loading the corresponding package (it affects output of Trace):

In[1]:= ClearAll[f, x]; f[x_] := x - 1;
Trace[FindRoot[f[x], {x, .5}, Evaluated -> False, Compiled -> False], 
  TraceInternal -> True] // LeafCount
Trace[FindRoot[f[x], {x, .5}, Evaluated -> True, Compiled -> False], 
  TraceInternal -> True] // LeafCount

Out[2]= 181

Out[3]= 111

In[4]:= ClearAll[f, x]; f[x_?NumericQ] := x - 1;
Trace[FindRoot[f[x], {x, .5}, Evaluated -> False, Compiled -> False], 
  TraceInternal -> True] // LeafCount
Trace[FindRoot[f[x], {x, .5}, Evaluated -> True, Compiled -> False], 
  TraceInternal -> True] // LeafCount

Out[5]= 217

Out[6]= 261

One can see that in simple cases the option Evaluated -> True reduces number of evaluations but in more complicated cases (black-box function) it cannot help.

edited Jun 12 '12 at 13:57

answered Jun 09 '12 at 06:53

Alexey Popkov

61,809
7
149
368

Evaluated is not listed as one of the options for FindRoot in the documentation. What does it do? – Michael Wijaya Jun 09 '12 at 06:58
5

It controls whether the first argument will be evaluated before the moment when the local variable (in this case x) will have concrete value assigned. See this Mathgroups thread for more information. – Alexey Popkov Jun 09 '12 at 07:03
7

It probably should be mentioned that the "standard" trick to provide a function which only evalutes for numeric arguments (f[x_?NumericQ] := (Print[x]; Sin[x] - Cos[x])) will also do the trick without the need of the undocumented option. – Albert Retey Jun 09 '12 at 09:31
It also works for Plot. Nice one! +1 – Rojo Jun 09 '12 at 14:06
@AlbertRetey I think you should post an answer with that. May help others. – Dr. belisarius Jun 09 '12 at 14:08
@AlbertRetey I am not sure I understand how that trick works in this context. Could you post an answer based on that? – Michael Wijaya Jun 09 '12 at 18:11
@Michael I updated the answer with the "standard" technique mentioned by Albert Retey. – Alexey Popkov Jun 10 '12 at 07:19
@AlexeyPopkov Thanks! Could you also include a link to the documentation where that "standard" trick is explained? I apologize for making so many requests of you. – Michael Wijaya Jun 10 '12 at 07:30
1

@Michael This technique is demonstrated in the Documentation in few places, for example here. – Alexey Popkov Jun 10 '12 at 08:01
@AlexeyPopkov I only wished all, and not just the first argument could be evaluated by Evaluated. Then Plot etc. could be constructed easier programatically. – Rolf Mertig Jun 11 '12 at 22:40
@Rolf Please compare outputs of Clear["`*"]; x = y; FindRoot[x, {y, .5}] and Clear["`*"]; y = x; FindRoot[x, {y, .5}]. You will see that in the first case everything is OK but in the second case FindRoot cannot understand that y==x. It means that FindRoot does not evaluate the second argument even with default option Evaluated -> True. Note also that these functions (Plot, FindRoot etc.) have attribute HoldAll. – Alexey Popkov Jun 11 '12 at 23:28
@AlexeyPopkov sure. I just wondered why this is the case. It should not, unless there is a good reason. Maybe WRI can still change the behaviour before it gets documented? – Rolf Mertig Jun 12 '12 at 08:28
@Rolf Now I see you point. I think that current behavior with not evaluating the second argument is correct and expected: it is mentioned in the Documentation that "FindRoot first localizes the values of all variables, then evaluates f with the variables being symbolic, and then repeatedly evaluates the result numerically." It means that the second argument (at least the variable istself) must not be evaluated because it should be localized: "FindRoot has attribute HoldAll, and effectively uses Block to localize variables." So I am sure that this aspect will not be changed. – Alexey Popkov Jun 12 '12 at 13:26
@Rolf I updated the answer with additional comparisons. – Alexey Popkov Jun 12 '12 at 13:58

Michael E2 · Answer 2 · 2017-10-01T13:12:26.730

First, it's probably worth adding to @Rojo's comment that EvaluationMonitor was introduced in V5, so that the functionality of Wagner's code can be achieved with

FindRoot[Sin[x] - Cos[x], {x, .5}, EvaluationMonitor :> Print[x]]

Second, note that using Evaluated -> False or ?NumericQ prevents symbolic analysis of the function and prevents FindRoot from choosing Newton's method, as is done in Wagner's book. In his book, the output of Print is

To get the essential part of Wagner's output with Evaluated -> False or a ?NumericQ function supply the Jacobian:

FindRoot[Print[x]; Sin[x] - Cos[x], {x, .5}, Evaluated -> False, 
 Jacobian -> D[{Sin[x] - Cos[x]}, {{x}}]]
(*
  0.5
  0.793408
  0.785398
  0.785398

  {x -> 0.785398}
*)

Note the initial x is missing, but otherwise the steps are the same.

Just because I like generalizing processes: Here's a generic wrapper, with a debugging hook debug that can be set to Print temporarily with Block.

ClearAll[dbFunc];
f[x_] := Sin[x] - Cos[x];
dbFunc[f_][x_?NumericQ] := (debug[x]; f[x]);
Derivative[1][dbFunc[f_]][x_] := f'[x]; (* need to Clear[Derivative] to undo this def *)
Block[{debug = Print},
 FindRoot[dbFunc[f][x], {x, .5}]
 ]
(*
  0.5
  0.793408
  0.785398
  0.785398

  {x -> 0.785398}
*)

Wagner's trick to monitor FindRoot no longer works in Mathematica version 8

2 Answers2

Another approach

Additional comparisons

Linked

Related