Parallelizing nested for loops

Question

I'm currently computing nested for loops.

I count up a few counting variables and compute a value for each set of counting variables, then that value is added to an array. It is a way of adding up the contributions of each of these counting variables.

It is something like this simplified code:

For[h = 1, h < hmax, h++, count1 = ((200/(hmax))*h);

For[j = 1, j < jmax, j++, count2 = ((100/(jmax))*j) ;


Value = ((user_defined_function1[count1-count2])^(2))*user_defined_function2[count2];

IArray =ReplacePart[IArray, index -> (IArray[[index]] + Value)  ] 

]]

The issue is that as my calculation has gotten more precise, I have found that I need many more counting variables. My new code has 6 nested for loops, instead of the 2 shown in this example. The code is taking too long.

I want to parallelize the calculation, since I have access to a machine with many slow cpu cores.

My first attempt will be replacing the For[] with Paralleldo[]. I read that Paralleldo[] has some complications if you call functions inside the Paralleldo[]. I don't think I understood the complication completely, so I am asking if there will be a problem if I proceed with replacing the for loops with Paralleldo[].

I think I can also reformulate the problem using ParallelTable[], but then the result will be in a different format and I will have to figure out how to extract the meaningful result. This extra work would not be ideal..

Can I use Paralleldo[] for this application? Are there problems? Are there better alternatives?

Thanks!

Answer 1

Looking at your code, it has a similar structure to this small example:

arr = ConstantArray[0, 5];
Do[
 Module[{index, value},
  index = Mod[i, 5, 1];
  value = i^2;
  arr = ReplacePart[arr, index -> arr[[index]] + value]
  ],
 {i, 100}
 ]
arr

Here each iteration of the loop modifies the variable arr. The key to effective parallelization in Mathematica is not to modify the same value from different parallel threads.

The code could be restructure like so:

(* generate index -> value pairs *)
rules = Table[Mod[i, 5, 1] -> i^2, {i, 100}];

(* combine them *)
Normal@SparseArray[
  Normal[Total /@ GroupBy[rules, First -> Last]],
  {5}
]

The idea is to first generate the index -> value pairs in a Table that can be easily changed to a ParallelTable, then combine them only when the parallel computation has finished.

ParallelCombine follows this principle, though in practice I often use ParallelMap/ParallelTable, then combine manually.

---

P.S. A slightly simpler, but undocumented way to combine the results is

SetSystemOptions["SparseArrayOptions" -> "TreatRepeatedEntries" -> Total];
Normal@SparseArray[rules]
SetSystemOptions["SparseArrayOptions" -> "TreatRepeatedEntries" -> First];

It is very important that you don't forget to set this back to the default First, otherwise things may break.