2

I'm trying to understand why a simple do-loop takes substantially longer when done in parallel than sequentially:

count = 0;  
SetSharedVariable[count];  
AbsoluteTiming[Do[count++, {nn, 100}]]  
Print[count]  

Out[493]= {0.000176, Null} (\*Time in seconds*)   
Out[494]= 100 (\*The result*)

Whereas if I do

count = 0;  
SetSharedVariable[count];  
AbsoluteTiming[ParallelDo[count++, {nn, 100}]]  
Print[count]  

Out[497]= {0.391256, Null}  (\*Time in seconds*)  
Out[498]= 100 (\*The result*)

Could someone please explain why ParallelDo takes 0.39 seconds whereas Do takes only 0.00017 seconds?

Thanks!

brama
  • 898
  • 5
  • 18
MvP
  • 265
  • 1
  • 6
  • 2
    Take a look here: Why won't Parallelize speed up my code?. SetSharedVariable forces count to be always evaluated on the main kernel. Thus you gain nothing, but you do lose a lot: evaluation will keep switching between parallel kernels and the main kernel, adding a considerable overhead. – Szabolcs Jun 20 '14 at 14:57
  • 1
    To see what SetSharedVariable really does, evaluate SetSharedVariable[count] first, then ParallelEvaluate[Information[count]]. It's a very straightforward implementation. – Szabolcs Jun 20 '14 at 15:27
  • @Szabolcs, thanks for the helpful comment and link. So then, if I understand correctly, in order for a single variable to be updated by, say, a billion independent loops I'm better off using Do loops instead of ParallelDo loops because of the communication overhead? – MvP Jun 20 '14 at 15:42
  • 2
    Usually, yes. The best thing is not to use a variable that's changed by multiple threads, but this is not possible for every algorithm. When it is possible, it simplifies parallelization dramatically (in any language). SetSharedVariable is only useful if the variable access is infrequent and takes negligible time compared to other computations. I only found SetSharedVariable useful in very few cases. – Szabolcs Jun 20 '14 at 15:46

0 Answers0