Why does the kernel die suddenly in this script with ParallelMap?

Question

I have the following script involving ParallelMap, but the Kernel "dies" during evaluation on a high performance computer. Also, on my MacBookPro it dies sometimes after one hour, sometimes after eight (you can tell because there is no error, and all the variables are lost from memory, so change colour).

Why is this? Is it overloading the machine with some memory requirements?

ed[rg_] := Module[{ge, el},
   ge = GraphEmbedding[rg];
   el = EdgeList[rg];
   EuclideanDistance @@@ Map[ge[[#]] &, el, {2}]
   ];
addvert[gr_, coord_, range_] := Module[{pts, nl, lv, vv, ee},
  pts = GraphEmbedding[gr];
  vv = VertexList@gr;
  lv = Length@vv;
  nl = lv + # & /@ (Range@Length@coord);
  vv = Join[vv, nl];
  ee = Join[
    Flatten[Thread[# <-> 
         Nearest[Thread[pts -> Range[Length[pts]]], 
          coord[[# - lv]], {Infinity, range}]] & /@ nl], 
    EdgeList@gr];
  Graph[vv, ee, VertexCoordinates -> Join[pts, coord]]
  ]
(*addverts[g_,newverts_,range_]:=Fold[addvert[#,#2,range]&,g,newverts]\
;*)
findT[g_] := Module[{l},
   l = Length@VertexList@g;
   lfixd = (Length@fixdverts) - 1;
   GraphDistance[g, l - lfixd, 
      l - #] & /@ (Reverse@Range@(Length@fixdverts - 1) - 1)
   ];
fixdverts = 
  Developer`ToPackedArray[
   Join[{{.5, .5}}, 
    Table[{.5 + 1/2 Cos[t], .5 + 1/2 Sin[t]}, {t, 0, 2 Pi, Pi/16}]], 
   Real];
T = ConstantArray[0, {400, 2}];
For[i = 100, i <= 400, i++,
  Clear[graphs, n];
  r0 = 1/(5 (1 + i/20)); density = 7/(Pi r0^2); Ngraphs = 20000;
  Print["Starting r0 = ", N@r0, ", Euclidean Distance = ", 
   N@(1/2) (1/r0), ", Density = ", N@density, ", ExpDegree = ", 
   N@(density Pi r0^2)];
  n = RandomVariate[PoissonDistribution[density], Ngraphs];
  graphs = 
   ParallelMap[addvert[#, fixdverts, r0] &, 
    Table[RandomGraph[SpatialGraphDistribution[n[[k]], r0]], {k, 1, 
      Ngraphs}]];
  graphs = 
   ParallelMap[SetProperty[#, EdgeWeight -> ed[#]] &, graphs];
  Print[Ngraphs, " graphs built!"];
  T[[i, 1]] = N@(1/2) (1/r0);
  T[[i, 2]] = 
   Variance[
    1/r0 DeleteCases[Flatten[ParallelMap[findT, graphs]], Infinity]];
  Print["Variance = ", N@T[[i, 2]]];
  ];

See also https://mathematica.stackexchange.com/questions/102268/how-to-restart-computation-when-kernels-die — apg, Mar 16 '18 at 09:21
Why not remove lines one by one until it runs to completion? — Michael Stern, Mar 16 '18 at 10:48
It only fails when it runs for a long time, so I can't simulate the intensity over short periods. Also, I think the problems are in the two parallel map lines, so I Imagine if I remove either the problem will go away. — apg, Mar 16 '18 at 10:50

score 1 · Accepted Answer · answered Mar 19 '18 at 22:29

1

The memory usage is too high. See e.g. Using less memory with ParallelMap when modifying random geometric graphs

answered Mar 19 '18 at 22:29

apg

2,075
10
13

Why does the kernel die suddenly in this script with ParallelMap?

1 Answers1