Difficulty catching thrown errors in asynchronous tasks

Question

In this Question, Leonid explained that module variables can get garbage collected from a chain of asynchronous tasks unless their Temporary attribute is cleared either by calling ClearAll on them or by Holding them. This works for preserving a variable that refers to the function that creates asynchronous task. The next step is to create a variable that Catches a value Thrown inside an asynchronous task, but the same technique does not seem to work, as I illustrate below.

First, a synchronous version that works as I need. This has an internal synchronous task wrapped in a Catch. By default, the Catch returns a symbolic value that is Unique; that is the default, "OK" case. Internally to the Catch, the task bumps a state variable, checks that the state value is ok, prints it if so, throws it if not, and schedules the next iteration of the task up to a maximum state value. "Ok" in this contrived example means that state's value is less than 2; in a real application, it would be based on some unpredictable outcome, hence the whole reason for Catch. The following code implements the scheme above and works fine, printing 1 and then ERROR:

Module[{
  state = 0,
  runNextTask,
  catchResult,
  defaultCatchResult = Unique[]},
 runNextTask = Function[
   catchResult = Catch[
     If[state < 4,
       state = state + 1;
       If[state < 2, Print[state], Throw[state]];
       runNextTask[],
      {0.10}];
     defaultCatchResult];
   If[catchResult =!= defaultCatchResult, Print["ERROR"]]];
 runNextTask[]]

Now, I want to do the same asynchronously, attempting the following code, wherein the only differences to the synchronous versions are marked with comments:

Module[{
  state = 0,
  runNextTask,
  catchResult,
  defaultCatchResult = Unique[]},
 (* the following device removes the "Temporary" 
 attributes from "runNextTask" and "catchResult." *)
 ClearAll[runNextTask, catchResult];
 runNextTask = Function[
   catchResult = Catch[
     If[state < 4,
      (* the following invocation techniques make the
      body here asynchronous. *)
      StartScheduledTask@
       CreateScheduledTask[
         state = state + 1;
         If[state < 2, Print[state], Throw[state]];
         runNextTask[],
        (* This is the amount of time with which to
        separate asynchronous invocations. *)
        {0.10}]];
     defaultCatchResult];
   If[catchResult =!= defaultCatchResult, Print["ERROR"]]];
 runNextTask[]]

The problem, now, is that the Thrown value is not caught, as revealed in the message box:

1
Throw::nocatch: Uncaught Throw[2] returned to top level.

This isn't the end of the world, since at least the repeating chain of asynchronous tasks is interrupted and I can wrap the entire thing in another catch and make it behave well inside other code. But it's another case where I am uncomfortable proceeding when I don't understand what's going on. I expected one result and got another!

I will be grateful for advice, suggestions, solutions!

I'll be grateful for any advice, clues, solutions.

Answer 1

Why this does not work

The problem here seems to be that Catch can only catch exceptions thrown by some code down the same evaluation stack, corresponding to the same evaluation process. However, the asynchronous mechanism you use (based on CreateScheduledTask etc) induced a different evaluation at a specified time, with another evaluation stack, and at a later time. So, at any given iteration, your Catch is watching for the stack number n, while your exception, if thrown, will be in a stack number n+1 - in other words, your Catch is always one evaluation too early.

In order to catch your exceptions, you'd have to place Catch inside CreateScheduledTask (so that it watches the correct evaluation stack), which would largely destroy the purpose. Therefore, I'd suggest to communicate the state and status of your system (error or not) via flags, which are set inside your code in CreateScheduledTask, and checked in your function, rather than using exceptions. You can, of course, still use exceptions to short-circuit the failure inside the code in CreateScheduledTask, but not as a communication mechanism between that code and the function which creates it (but see also below).

Time travel and lazy evaluation, or how to make it work

However, using exceptions in the fashion you want is not impossible, thanks to the lazy evaluation, available in Mathematica. If you insist on minimal modification for your code, then here is how: make your function runNextTask accept some arbitrary code as a parameter, and hold it, evaluating only inside body. Then:

Module[{state = 0, runNextTask, catchResult,
    defaultCatchResult =  Unique[]},
    (*the following device removes the "Temporary" 
      attributes from "runNextTask" and "catchResult."*)
    ClearAll[runNextTask, catchResult];
    SetAttributes[runNextTask, HoldAll];
    runNextTask =
     Function[
        code,
        catchResult =
          Catch[
             code;
             If[state < 4,
              (*the following invocation techniques make 
                the body here asynchronous.*)
                 StartScheduledTask@
                   CreateScheduledTask[
                     state = state + 1;
                     runNextTask[If[state < 2, Print[state], Throw[state]]],
                     (*This is the amount of time with which to separate 
                      asynchronous invocations.*)
                     {0.10}
                   ]
             ]; (* If *)
             defaultCatchResult
          ]; (* Catch *)
          If[catchResult =!= defaultCatchResult, Print["ERROR"]],
          HoldAll
     ]; (* Function *)
     runNextTask[Null]]

Note that the function is now HoldAll, as well as the runNextTask (the latter isn't really necessary though). By using this construct, I basically pass the code from the "future" to the "past" (from the stack-matching perspective. Of course, it is really the other way around), making it evaluate in a different evaluation stack, now the same as that where Catch is - and now it works.