3

This was a wild head-scratcher for me to hunt down, but I now am sure LetL is the culprit. Here's a small example:

ParallelEvaluate[LetL[{id = $KernelID, sq = id*id}, sq]]

which gives

With: Variable 1 in local variable specification {1} requires a value.

With: Variable 2 in local variable specification {2} requires a value.

With: Variable 3 in local variable specification {3} requires a value.

General: Further output of With::lvws will be suppressed during this calculation

Out[1] = {With[{1}, With[{1}, 1]], With[{2}, With[{4}, 4]], With[{3}, With[{9}, 9]], With[{4}, With[{16}, 16]]}

A similar failure happens with 

ParallelTable[LetL[{x = i j}, x^2], {i, 1, 3}, {j, 1, 3}]
ParallelMap[LetL[{f = #^2}, Sqrt[f]] &, Range[1, 10]]

Question: Can we formulate LetL in a way that it performs correctly inside a ParallelEvaluate, ParallelTable, and other Parallel functions?

Partial answer: Put LetL in its own package and evaluate ParallelNeeds["LetL`"]. Then the above examples work correctly. I use LetL enough that I have it in my auto-loaded packages, and I don't always want to launch multiple kernels on startup.

Alternative question to solve my use case: How can I get LetL to be ParallelNeedsed when additional kernels are launched?

evanb
  • 6,026
  • 18
  • 30
  • Nowadays, you can do this: With[{a = 1}, {b = a}, b]. Although the syntax highlighting is still incorrect, that might work for you. At least ParallelEvaluate[With[{id = $KernelID}, {sq = id*id}, sq]] does what I expect. – Henrik Schumacher Mar 23 '20 at 17:04
  • @HenrikSchumacher I verified that this works, but it's also completely undocumented (I'm on 12.1). It also would require me hunting down every example of LetL in my code :P – evanb Mar 23 '20 at 17:47
  • 2
    I can't reproduce the problem, tried on 12.0 and 12.1 - in both cases things work for me just fine without errors (Mac OS X 10.13) – Leonid Shifrin Mar 23 '20 at 18:03
  • Curious. I actually use the code here and link mathematica/Let.wl into $UserBaseDirectory/Applications. I have Needs["Let`"]; in my $UserBaseDirectory/Kernel/init.m. That's all that goes into my example. – evanb Mar 23 '20 at 22:22

1 Answers1

3

The problem

Apparently, the problem is not with Let itself, but with the fact that the symbol's definitions are not autoloaded, as you noted youself . So what happens then is that the HoldAll attribute of Let is being ignored by the sub-kernels, and when you get the result back for evaluation on the master kernel, it is too late.

Possible solution

What I will suggest below may be a bit heavy-handed, but it should work. You can convert the code into a paclet. The repository with working code is here, where you can look for details of how to set things up so the symbol gets autoloaded.

You can download and extract the contents into applications folder, but I have also added a bootstrap installer which would do it for you, for convenience. The following code needs to be executed just once (note the call to Quit at the end):

Import[
   "https://raw.githubusercontent.com/lshifr/LetWL/master/BootstrapInstall.m"
]

RebuildPacletData[]

Quit

After which it works just fine:

ParallelEvaluate[LetWL`Let[{id = $KernelID, sq = id*id}, sq]]

(* {1, 4, 9, 16} *)

Note that I have changed the context of Let to be LetWL`. Otherwise, the code of implementation of Let is pretty much the same as your original one.

Leonid Shifrin
  • 114,335
  • 15
  • 329
  • 420
  • Is the key essentially "AutoloadSymbols" -> {"LetWLLet"},inLetWLLoader.m`? – evanb Apr 01 '20 at 16:05
  • @evanb Yes, it is. I don't remember now what role and importance of Symbols property in PacletInfo.m, and whether or not you can omit that. – Leonid Shifrin Apr 01 '20 at 16:15