32

It is not completely clear to me how Return[] works.

The documentation says:

Return[expr] returns the value expr from a function.

But in Mathematica it is not clear where are the boundaries of a "function" defined using patterns and Set(Delayed). The system simply applies replacement rules until there's nothing to change anymore.

Consider these examples:

a := (Module[{}, Return[0]; 1]; 2)    
a
(* 0 *)

It is not completely clear to me why Return breaks out from a, and not from Module or one of the CompoundExpression.

The following doesn't work, and simply returns the Return expression. It reminds me a bit of how Unevaluated is handled.

(Module[{}, Return[0]; 1]; 2)
(* Return[0] *)

If we add another layer of definitions, the effect of Return will stop there:

b := (a; 3)
(* 3 *)

Why do these three inputs give different results? What is the general rule for deciding where Return breaks out to precisely?

The answer lies somewhere in fully understanding the evaluation process.

Would someone care to elucidate this point, perhaps with pointers to the parts of the documentation which make this clear?

Alexey Popkov
  • 61,809
  • 7
  • 149
  • 368
Szabolcs
  • 234,956
  • 30
  • 623
  • 1,263
  • 4
    Be aware that the behavior of Return has changed from one version to another. I usually try to avoid using it myself. – Mr.Wizard Dec 16 '11 at 14:19
  • 2
    @Mr.W Wow, if its behaviour changes even between minor versions, than indeed it's better avoided! – Szabolcs Dec 16 '11 at 14:24
  • 2
    @Mr.Wizard, Szabolcs Personally I don't consider this particular observation as a strong evidence against using Return, since it is just the addition of new functionality (Fold), and no old code is broken (see also my commens below my answer). – Leonid Shifrin Dec 16 '11 at 15:20
  • 1
    @Leonid I seem to recall at least one other case, on Mathgroup I suppose, where Return behaved differently per version. Does that sound unfamiliar to you? Also, I haven't given this a lot of thought but couldn't the change re: Fold break existing v7 code? It would probably be an odd use of Return, but not as odd a mutable PatternTest functions... ;-) – Mr.Wizard Dec 16 '11 at 15:33
  • 1
    @Mr.Wizard Well I guess one can construct an example of code that would be "broken" in a new version, but I have a hard time imagining that it would be less perverse than the method you mentioned (which, actually, I did use constructively several times). – Leonid Shifrin Dec 16 '11 at 16:31

1 Answers1

30

The exhaustive answer to this has been given in this thread (if you combine several answers there. The mentioned thread is also a generally recommended read for more information on this - at least one I am aware of). I will first reproduce here those bits of my answer from there which were correct.

There are two possible outcomes for any expression wrapped in Return: either it is inside some lexical (or dynamic) scoping construct for which the action of Return is defined - and then the presence of Return will lead to breaking-out-of-the-scoping-construct procedure, or it is not and then it is just a symbolic expression like any other. After breaking out of the scoping costruct (or just evaluation if it was not inside any scoping construct), Return gets discarded only if it was called from within the r.h.s. of a user-defined rule. Here is an example: 

In[1]:= 
Clear[a,b,c]; 
c=(Return[a];3) 

Out[1]= Return[a] 

In[2]:= b:=(Return[a];3) 
In[3]:= b 
Out[3]= a

This behavior can be explained by consulting the exact rules of the evaluation procedure. Lacking a more up-to-date account, I cite here David Withoff's "Mathematica internals" of 1992: The very last step of the evaluation loop is (Chapter 3 - evaluation, p. 7, on the bottom): "Discard the head Return, if present, for expressions generated through application of user-defined rules."

Thus, when you use SetDelayed, you create a user-defined delayed rule and then Return is discarded, while for "direct" evaluation like 

In[4]:= Return[a] 
Out[4]= Return[a]

it is not. But, in 

In[5]:= 
ClearAll[a,c];
a:=Return[c];
a

Out[7]= c

it is, and Return is discarded, even though there was no scoping construct to break out of. The same happens in your example where Module is wrapped around - Return breaks out of Module all right, but is not discarded since Module was not the r.h.s of any user-defined rule.

Here are the best descriptions of how Return works that I am aware of, due to Alan Hayes (can be found in this MathGroup thread) :

"In view of the recent thread on Holding Arguments in a Second Argument List the following notes on Return may be of interest:

  • If Return[x] is generated as a value in Do or Scan then x is immediately returned;

  • If Return[x] is generated as an entry in CompoundExpression or as a value of the body of a While or For loop then Return[x] (not x) is immediately returned;

  • If Return[x] is generated as the value of a user-defined function then the function returns x (not Return[x])

  • Otherwise Return[x] behaves as a ordinary expression. "

These "empirical" rules agree with the statement found in the Withoff's report, the latter making the former sound somewhat less magical.

It is also good to remember that Return has a second optional argument, which tells it from which surrounding scoping construct to break out:

In[242]:= (Module[{}, Return[0, CompoundExpression]]; 2)
Out[242]= 0

but

In[243]:= (Module[{}, (Return[0, CompoundExpression]; 0)]; 2)
Out[243]= 2

In your last example, the outer CompoundExpression defined in b in dynamically rather than lexically scoped (relative to the stuff defined in a), and therefore this won't work (in other words, your last result is also as expected).

Leonid Shifrin
  • 114,335
  • 15
  • 329
  • 420
  • Note that I edited my original answer quite a bit. My answer from the Mathgroup thread was partially misleading, so I modified some parts of it. – Leonid Shifrin Dec 16 '11 at 13:21
  • 1
    Let me see if I got it right: 1. Module/Block/etc. do not modify the handling of Return (i.e. they're just the same as CompoundExpression: the rest of the CompoundExpression is ignored and Return[value] is returned immediately). 2. Do and Scan do behave specially, as they return value and not Return[value]. 3. Return[value] is always replaced by value if it is the result of the application of a delayed-type replacement rule (:=), but not in the case of a normal replacement rule (=) ... – Szabolcs Dec 16 '11 at 13:23
  • 2
    ... Another interesting point that I didn't expect is the difference between := and =. Try a := Return[0] and b = Return[0]. The two behave differently, but their OwnValues look exactly the same (both use :>). Information does show us if := or = was used in the definition though. I was not aware that there existed a difference in the handling of := and = even after the definition was made ... – Szabolcs Dec 16 '11 at 13:25
  • ... finally, is this part of your book correct? It says: "If we use Return[] however, we will also break from the entire localizing construct which encloses the loop (if there are nested localizing constructs, we break from the innermost one only)." (emphasis by me). It doesn't appear to be the case that Return breaks from the innermost only. Instead where it breaks to seems to be determined by the user-defined delayed rules (:=). Is this correct? – Szabolcs Dec 16 '11 at 13:27
  • 1
    @Szabolcs Your comments do indicate that I did not put enough effort in my answer, which is true - I mostly compiled it the sources I knew :). Ok, for your first comment: all correct, but the reason for direct assignment is not that this is not a replacement rule, but that its r.h.s. was evaluated immediately and that evaluation was not caused by an existing replacement rule. Consider, for example: ClearAll[a, b, c];a = b;b = Return[c];a - you get plain c at the end, even though all rules were immediate - but here Return[c] from b was a result of evaluation of the rule for a – Leonid Shifrin Dec 16 '11 at 13:35
  • @Szabolcs as for my book, the statement was correct I think (although admittedly I was not aware of all this stuff yet when writing it): as I mentioned in this answer, there are 2 distinct stages (or at least this is my picture of it): 1. Breaking out of a scoping construct (and this is what my book statement was about), and 2: deciding whether or not to discard Return head - this is where the presence or absence of user-defined rules matters. – Leonid Shifrin Dec 16 '11 at 13:38
  • 1
    Yes, you are right. For some reason I thought that after a = Return[0] evaluating a would return Return[0] which is not the case. It return 0. So there's no difference between := and = after the definition has been made after all. Some more comments on Return: 1. f[Return[0,f]] does not behave the same way as Module[{}, Return[0,Module]] (or Fold or apparently any other builtin from where it might make sense to break out). The second argument of Return is not documented in M8 though, so perhaps it deprecated. 2. Just a note that Return will return from a Dialog[] – Szabolcs Dec 16 '11 at 13:45
  • To clarify: Based on the description in your book, I constructed x = Module[{}, Module[{}, While[True, Return[0]]; 1]; 2] It returns 0, not 2, i.e. it breaks from both Modules, not only the innermost one, as the book states. Do, however, is different from While: it breaks from the Module and not only from Do, and it breaks from the innermost Module only: x = Module[{}, Module[{}, Do[Return[0], {10}]; 1]; 2]. For behaves like While, not like Do (i.e. it's not special). Can you verify again please? – Szabolcs Dec 16 '11 at 14:10
  • See Mr.Wizard's comment above, Return seems to be unreliable, and it sbehaviour has changed even between minor versions. Most likely it was different at the time you wrote the book than in 8.0.4. – Szabolcs Dec 16 '11 at 14:26
  • 1
    @Szabolcs Ok, so: First, it seems that Return with a second argument only works for certain enclosing functions, not just any arbitrary f. Second, while Return with two arguments isn't documented, it is unlikely to go away and is "semi-officially" endorsed (see e.g. http://stackoverflow.com/questions/7446032/searching-from-the-end-of-a-list-in-mathematica/7446843#7446843), so I'd consider it safe to use. Next: the case you constructed: x = Module[{}, Module[{}, While[True, Return[0]]; 1]; 2] - here it appears that Return breaks from both Module (it does), but the mechanism is .. – Leonid Shifrin Dec 16 '11 at 14:59
  • 1
    @Szabolcs ...different from what it may look. It first only breaks from the innermost one. But then, Return appears inside the CompoundExpression in the outer (since Return is not discarded at that stage), then it quits the outer Module too. Consider e.g. this:Module[{}, q@Module[{}, While[True, Return[0]]; 1]; 2], which only breaks from the inner-most one, since the content of outer Module now is (q[Return[0]];2), and since Return is not the top-level element of CompoundExpression, it is ignored by it (it does not even come to Module). Regardless of the explanation, ... – Leonid Shifrin Dec 16 '11 at 15:08
  • @Szabolcs this looks like a wrong behavior to me, but this sheds some light on how the breaking-out mechanism is implemented: any of the constructs (such as CompoundExpression, Module, etc) will stop upon hitting Return, and return this Return - very simple idea. As for Do, it does not break from anything except itself (because Do is a dynamic scoping construct), just as Scan (because Scan has a special semantics regarding Return, which is intended). As to the version differences, it looks like Fold was added to a list of "Return-aware" constructs recently, but this ... – Leonid Shifrin Dec 16 '11 at 15:13
  • @Szabolcs does not make Return generally unreliable IMO (I don't have any other evidence of version inconsistencies regarding Return, and in any case the addition of Fold does not break any old code, just adds new possibilities). Your observation about the Dialog is nice. – Leonid Shifrin Dec 16 '11 at 15:15
  • @Szabolcs Upon some reflection - you were right that my statement in the book regarding nested Module-s wasn't correct - Return does break from all of them (as we discussed). For example, this f[x_] := Module[{y = x}, Module[{}, Return[y]; 1]; 2]; f[3] returns 3, not 2. I actually think that this behavior is erratic. And the fix is rather unexpected (to me anyways): f[x_] := Module[{y = x}, Module[{}, Return[y, Module]; 1]; 2]; f[3] returns 2. But you know what - I don't feel particularly guilty - the rules for Return are very peculiar (to say the least), and barely documented. – Leonid Shifrin Dec 16 '11 at 20:45
  • An alternative fix here would be catchReturn[x_] := x; f[x_] := Module[{y = x}, catchReturn@Module[{}, Return[y]; 1]; 2];. Here f[3] also evaluates to 2 – Jacob Akkerboom May 05 '14 at 13:44
  • We can also use the anonymous function If[Head[#] === Return, #[[1]], #] & (of course?). – Jacob Akkerboom May 05 '14 at 13:50
  • @JacobAkkerboom Sure, this will work, but the whole point is that one shouldn't be bothered to use such extra things, since it is quite easy to forget using them. – Leonid Shifrin May 05 '14 at 14:03
  • @LeonidShifrin that makes sense. I just wanted to give an alternative to Return with 2 arguments :). – Jacob Akkerboom May 05 '14 at 14:11