Why doesn't my expression evaluate at the subkernel?

Question

Consider this code:

BeginPackage["Test`"];
f1::usage = "";
f2::usage = "";
f3::usage = "";
Begin["`Private`"];
fQ[myf_] := Head[myf] === f1;
f1[n_][t_] := 0;
f2[myf1_?fQ, myf2_?fQ][t_] := 0;
f3[myf1_] := myf1[0.];
End[];
EndPackage[];

The expression evaluate on main kernel

f3[f2[f1[1], f1[1]]]
(*0*)

but when I evaluate on subkernels, it seems that it is not evaluated

LaunchKernels[];
DistributeDefinitions[f1, f2, f3];

ParallelEvaluate[Block[{data}, data = f3[f2[f1[1], f1[1]]];
  Print[data];], (Kernels[])[[1]]]

(* (kernel 1) Test`f2[Test`f1[1],Test`f1[1]][0.]*)

Where do I do wrong?

And if I distribute the definition of the fQ, it then get evaluated:

DistributeDefinitions[f1, f2, f3, Test`Private`fQ];
ParallelEvaluate[Block[{data}, data = f3[f2[f1[1], f1[1]]];
  Print[data]], (Kernels[])[[1]]]
(*  (kernel 1) 0 *)

How should I understand this behavior?

Somebody more knowledgeable than me will probably have a better answer, but it looks as though the parameter testing for f2 needs to call fQ, and that function definition isn't automatically distributed to the other kernels. — image_doctor, Jan 31 '15 at 10:34

score 4 · Accepted Answer · answered Jan 31 '15 at 16:05

As @image_doctor notes in a comment, the problem occurs because the definition of Test`Private`fQ is not being distributed to the other kernels.

DistributeDefinitions uses Language`ExtendedFullDefinition to determine which definitions to transmit. We can see this by evaluating the following expression:

On[Language`ExtendedFullDefinition]
DistributeDefinitions[f1, f2, f3];
Off[]

(* ...
   Language`ExtendedFullDefinition::trace :
   Language`ExtendedFullDefinition[{f1,f2,f3},ExcludedContexts->{Algebra,...}] -->
   Language`DefinitionList[
      f1->{...,SubValues->{HoldPattern[f1[<<1>>][<<1>>]]:>0},...},
      f2->{...,SubValues->{HoldPattern[f2[<<2>>][<<1>>]]:>0},...},
      f3->{...,DownValues->{HoldPattern[f3[<<1>>]]:>Test`Private`myf1[0.]},...}]. >>
*)

Test`Private`fQ is absent from the generated Language`DefinitionList.

Language`ExtendedFullDefinition is normally able to trace all dependencies in a function:

ff[x_] := gg[x]
gg[x_] := x + 1
Language`ExtendedFullDefinition[ff]

(*
   Language`DefinitionList[
     ff->{...,DownValues->{HoldPattern[ff[x_]]:>gg[x]},...},
     gg->{...,DownValues->{HoldPattern[gg[x_]]:>x+1},...}
   ]
*)

But apparently it does not account for pattern tests in definitions:

ff2[x_?gg2] := 1
gg2[x_] := True
Language`ExtendedFullDefinition[ff2]

(*
   Language`DefinitionList[
     ff2->{...,DownValues->{HoldPattern[ff2[x_?gg2]]:>1},NValues->{},...}
   ]
*)

The pattern test function gg2 is missed by the dependency scan. This would appear to be a bug, but I might just be unaware of some compelling reason to ignore pattern tests in these circumstances.

As noted in the question, the work-around is to manually identify and distribute any missing dependencies. The "distribute" part of that is easy -- it is the "identify" part that can be difficult.

Thanks for the answer. It appears to me like a bug too. Otherwise, one have to manually distribute all the private definitions. I have reported as a bug to technical support. — xslittlegrass, Jan 31 '15 at 16:59

Why doesn't my expression evaluate at the subkernel?

1 Answers1

Linked