4

Consider a symbol in System such as CircleDot. I want to redefine this symbol and distribute the definition to subkernel, to use with ParallelEvaluate or the likes.

With a standard symbol in Global I can simply do

foo[___] := 1;
Print @ DownValues @ foo
ParallelEvaluate[Print @ DownValues @ foo];

which will show that the definition is automatically distributed to subkernels.

However, doing the same with CircleDot doesn't work:

enter image description here

Using DistributeDefinitions@CircleDot doesn't seem to change this. Why does this happen?

I know I can force subkernels to accept the definition by directly doing something like ParallelEvaluate[CircleDot[___]:=1], but this might not be desirable in some cases, and also I would like to understand why some symbols seem to behave different in this context.

glS
  • 7,623
  • 1
  • 21
  • 61
  • Does this answer your question? How are function definitions distributed to parallel kernels? – Szabolcs Sep 14 '21 at 19:32
  • See also https://mathematica.stackexchange.com/a/128287/12 – Szabolcs Sep 14 '21 at 19:32
  • @Szabolcs thanks, hadn't seen those. So a possible solution is changing $DistributedContexts. This is however in this case not desirable, as far as I can tell, as we don't want to let the whole of System` distribute. Is there a way to have DistributeDefinitions work on a specific function in System`? The solution given in the linked post, DistributeDefinitions[const`precisionRate], doesn't seem to work in this case: DistributeDefinitions[System`CircleDot] doesn't do what it should – glS Sep 14 '21 at 19:41
  • I was going to say that the solution is not to change $DistributedContexts globally, but to change the DistributedContexts option of value in a single DistributeDefinitions call when distributing CircleDot. However, I just tried this and it does not work. It seems there is a hard limitation on System symbols. I am not surprised, as distributing the wrong ones could cause trouble, and DistributeDefinitions does indeed distribute dependent symbols recursively. – Szabolcs Sep 14 '21 at 19:58
  • Thus what I'd do is just to re-issue the definition on the subkernels with ParallelEvaluate: ParallelEvaluate[CircleDot[___]:=1]. A better solution is to put the CircleDot definition into a package (eventually you'll end up doing this anyway) and load that package using ParallelNeeds. – Szabolcs Sep 14 '21 at 19:59
  • I'm sorry. I retracted the close vote, and I'm going to sleep on this. I'm just too tired tonight. – Szabolcs Sep 14 '21 at 20:00
  • 2
    In trying to find an automatic way to identify the symbols that need to be redefined in each parallel kernel, I find it weird that Language\ExtendedFullDefinition[System`CircleDot]is an empty definition list, even after being defined. In contrast,Definition[CircleDot]andInformation[CircleDot, "Definitions"]` shows the user definition. Thought i'd mention it. – thorimur Sep 14 '21 at 22:24
  • 1
    @thorimur to see the definition list you have to use Language`ExtendedFullDefinition[CircleDot, "ExcludedContexts" -> {}] (see my answer below) – Hausdorff Nov 26 '21 at 14:30

1 Answers1

6

Some spelunking of DistributeDefinitions shows that all contexts listed in Parallel`Protected`$ExcludedContexts are by default excluded from being distributed. In particular, Parallel`Protected`DistDefs has the line

updates = ExtendedFullDefinition[expr, "ExcludedContexts"->$ExcludedContexts]

So to make sure that new definitions of symbols in System` are also distributed you can use something like

CircleDot[___]:=1;

Block[
  {ParallelProtected$ExcludedContexts=
    DeleteCases[ParallelProtected$ExcludedContexts,"System`*"]},
  DistributeDefinitions[CircleDot];
];


ParallelEvaluate[Print @ DownValues @ CircleDot];

{HoldPattern[CircleDot[___]]:>1}
{HoldPattern[CircleDot[___]]:>1}
{HoldPattern[CircleDot[___]]:>1}
{HoldPattern[CircleDot[___]]:>1}

This way we also only distribute new definitions for CircleDot and nothing else in System`.

Regarding the comment of @thorimur: you do not get the full definition of CircleDot from Language`ExtendedFullDefinition, as the default option "ExcludedContexts"->Automatic seems to filter System` as well. So by setting this option yourself you can also see the full definition

Language`ExtendedFullDefinition[CircleDot, "ExcludedContexts" -> {}]

LanguageDefinitionList[HoldForm[CircleDot] -> {OwnValues -> {}, SubValues -> {}, UpValues -> {}, DownValues -> {HoldPattern[CircleDot[___]] :> 1}, NValues -> {}, FormatValues -> {}, DefaultValues -> {}, Messages -> {}, Attributes -> {}}, HoldForm[BlankNullSequence] -> {OwnValues -> {}, SubValues -> {}, UpValues -> {}, DownValues -> {}, NValues -> {}, FormatValues -> {}, DefaultValues -> {}, Messages -> {}, Attributes -> {Protected}}, HoldForm[Rule] -> {OwnValues -> {}, SubValues -> {}, UpValues -> {}, DownValues -> {}, NValues -> {}, FormatValues -> {}, DefaultValues -> {}, Messages -> {HoldPattern[Rule::rhs] -> "Pattern1appears on the right-hand side of rule2`."}, 
Attributes -> {Protected, SequenceHold}}, HoldForm[List] -> {OwnValues -> {}, SubValues -> {}, UpValues -> {}, DownValues -> {}, 
NValues -> {}, FormatValues -> {}, DefaultValues -> {}, Messages -> {}, Attributes -> {Locked, Protected}}]
Hausdorff
  • 3,505
  • 1
  • 9
  • 25