Module variable scoping in Scheduled Tasks?

Question

I ran across a scoping puzzle while experimenting with ScheduledTasks, and I'd be grateful for an explanation from the sages here. I found a workaround by intuitive horse sense, but was unable to explain to myself adequately what was wrong with my original attempt.

First, a straightforward, tail-recursive, synchronous Module

Module[{state = 0, doNextIteration},
 doNextIteration = Function[
   state = state + 1;
   If[state < 4,
    (Print[state]; doNextIteration[])]];
 doNextIteration[]]
(* 1 2 3 *)

Next, my first attempt to do the same with asynchronous tasks, which I need to tail-chain as one-shot ScheduledTasks. If CreateScheduledTask has a second List argument, then it creates a one-shot task delayed by the time in seconds in the list, just what I need for my bigger application. In this simplified sample, the ScheduledTask tail-chains a new one-shot ScheduledTask by a tail-recursive call -- just like the successful synchronous code -- once every tenth of a second until the condition on state produces False (ignore cleanup of the task objects for simplicity, here).

Module[{state = 0, runNextTask},
 runNextTask = Function[
   StartScheduledTask@
    CreateScheduledTask[
     state = state + 1;
     If[state < 4,
      (Print[state]; runNextTask[])],
     {0.10}]];
 runNextTask[]]

OK, the problem is this only Prints once, not the desired three times. I had a sickening hunch that the problem has something to do with runNextTask being local to the Module and somehow not being able to refer to itself recursively -- even though there is only one Module -- in the bizzaro-land of asynchrony, so I "fixed" the code as follows:

Module[{state = 0, runNextTask = Unique[]},
 runNextTask[] :=
  If[state < 3,
   StartScheduledTask@
    CreateScheduledTask[
     Module[{},
      state = state + 1;
      Print[state];
      runNextTask[]],
     {0.10}]];
 runNextTask[]]

I made runNextTask refer to a Unique global symbol, and made my function into a rewrite rule attached to runNextTask[]. This works great, but I don't understand well why it does. I hate being as dumb as a horse, even if I can jump the fences.

Clues, advice, explanations: all appreciated.

Answer 1

You hit a rather subtle behavior, related to the garbage-collection and the Temporary attribute, and the semantics of Module regarding returning expressions. The thing is, to achieve your goal, you need the Module-generated variable (function)'s definition(s) to be exported outside Module (to be persistent). But, since you return not the symbol itself, but its r.h.s. (which is a chunk of code with delayed evaluation, either Function or ScheduledTaskObject in this case), the definitions of runNextTask are destroyed because it has a Temporary attribute, despite the fact that code containing this variable has been exported outside Module.

To illustrate my point, here are two work-arounds which both will lead to your desired effect:

1. Instead of assigning runNextTask to Unique[], insert a line ClearAll[runNextTask] right after the Module declarations. This will remove the Temporary attribute and the definition of runNextTask will persist.

2. Wrap your final call in Hold (so, return Hold[runNextTask[]]), and then use ReleaseHold outside Module.

This is your code for this option:

res = 
 Module[{state = 0, runNextTask},
   runNextTask[] :=
     StartScheduledTask@
       CreateScheduledTask[
         state = state + 1;
         If[state < 4,
           (Print[state]; runNextTask[])
         ],
         {0.10}];
   Hold@runNextTask[]]

ReleaseHold[res]

In this case, the definition also persists.

Note that this problem is not seen in examples where all evaluation happens at the time when Module is left: the following code will execute promptly

Module[{f, n = 0},
  f[] := (n++; If[n < 10, Print["*"]; f[]]);
  f[]]

even though all definitions for f are destroyed at the end - because the execution here is immediate (the same evaluation process, so that f still has all definitions during this evaluation), while delayed and asynchronous code induces a separate evauation process.