First come, first save in ParallelTable parallelization

Question

Short version of the question: In C++'s OpenMP, I know how to get what I want, but I'm wondering whether Mathematica has the same option. It's called "dynamic scheduling" when parallelizing.

Long version of the question: Say I'm evaluating a parametrized expression, MyFunc[i], where the evaluation time depends on i and is not constant. I would like to use my full computer power with ParallelTable:

result = ParallelTable[MyFunc[i], {i,1000} (*, Method -> [what should this be?] *)];

I would like to have ParallelTable evaluate MyFunc[i] on "first come first server" basis. In other words: Keep all Kernels busy as long as any instance of MyFunc[i] is not evaluated until the number of parallel kernels available (threads) is higher than the number of remaining evaluations of MyFunc[i].

I noticed that Mathematica decides how the parallelization is done before the evaluation. This also seems to be the case in the documentation. All the options provided for Method-> decide on how to split the list, not how to evaluate the list.

So: Is there a way to get the parallelization as "first come first serve"?

Answer 1

I'm not a expert, but it seems that ParallelSubmit does what you need.

Here is a little program that has behavior you describe :

ParallelTable[
     Print[i, " ", $KernelID]; 
 If[i == 1, Pause[4], Pause[1]];
 $KernelID,
{i, 1, 6}]

As the evaluation of the first item (i=1) needs 4 seconds and the evaluation of the other ones needs 1 second, the kernel that execute the first item should be used only once. This is not the case.

If you use the following code :

h1 = Table[
       ParallelSubmit[{i},
         Print[i, " ", $KernelID]; 
         If[i == 1, Pause[4], Pause[1]];
         $KernelID],
    {i, 1, 6}]
WaitAll[h1]

Then you have :

The kernel that executes the item 1 is used once.

This is also visible in real time on the panels of h1.

EDIT

It turns out that if the option Method -> "FinestGrained" is added to ParallelTable , it works fine too. Finally, I'm not sure the ParallelSubmitis usefull for the OP's problem :

ParallelTable[Print[i, " ", $KernelID];
 If[i == 1, Pause[4], Pause[1]];
 $KernelID, {i, 1, 6}, Method -> "FinestGrained"]