6

For some reason, I'd like to use ParallelTable with a variable number of iterators.

  • Table[a[1], {a[1], 0, 10}] works fine:

Output: {0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10}

  • ParallelTable[a[1], {a[1], 0, 10}] returns an error:

Output: ParallelTable::nopar1: ParallelTable[ci[1],{ci[1],0,10}] cannot be parallelized; proceeding with sequential evaluation. >>

{0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10}

I am also interested in workarounds to use ParallelTable and a variable number of indices (typically: ParallelTable[f[Table[a[i],{i,1,n}]],Table[a[j],{j,1,n}]] where f is a function of a list with variable size).

Community
  • 1
anderstood
  • 14,301
  • 2
  • 29
  • 80
  • It appears ParallelTable[a[1], {a[1], 0, 10}] not working with variable like a[1], since ParallelTable[f[i], {i, 0, 10}] does work. `ParallelTable[f[Table[a[i],{i,1,n}]],Table[a[j],{j,1,n}]] does not make sense. – Kattern Jun 06 '15 at 04:54
  • @kattern: Yes, that's my question: why does it work with Table but not with ParallelTable.

    With[{iter = Sequence @@ Table[{a[j],-1, n}, {j, 1, n}]}, Table[f[Table[a[i], {i, 1, n}]], iter]] if you prefer...

     – anderstood Jun 06 '15 at 04:56
  • Still do not understand, can you please write an executable Table version of what you what? – Kattern Jun 06 '15 at 05:01
  • @kattern Try this standalone code (which works): With[{iter = Sequence @@ Table[{gag[j], -1, 1}, {j, 1, 3}]}, Table[f[Table[gag[i], {i, 1, 3}]], iter]]!. How can I make it work with ParallelTableinstead of Table? – anderstood Jun 06 '15 at 05:02
  • @kattern: Well... changing Table to ParallelTable works in this case. I still don't get it, why does ParallelTable[a[1], {a[1], 0, 10}]not work? – anderstood Jun 06 '15 at 05:05
  • Yes, quite strange. Maybe you can substitute the variable like ReleaseHold[Hold[ParallelTable[a[1], {a[1], 0, 10}]] /. a[1] -> i] first. Let wait to see whether there is a better solution. – Kattern Jun 06 '15 at 05:11

2 Answers2

6

Using Trace we can see that the evaluation of ParallelTable[a[1], {a[1], 0, 10}] becomes:

Parallel`Combine`Private`parallelIterateE[
  ParallelTable, Table, Join, Identity, a[1], {a[1], 0, 10}, {Automatic, "Global`"}
]

Further using PrintDefinitions in the GeneralUtilities package lets us peek behind the curtain:

Needs["GeneralUtilities`"]

PrintDefinitions @ Parallel`Combine`Private`parallelIterateE

We see definitions specifically wanting a Symbol:

Parallel`Combine`Private`parallelIterateE[orig_, iter_, comb_, f_, 
  expr_, {i_Symbol, vals_List}, others___, {meth_, dist_, ___}] := . . .

Parallel`Combine`Private`parallelIterateE[orig_, iter_, comb_, f_, expr_, 
  it : {i_Symbol, w0_ : 1, w1_, dw_ : 1}, others___, {meth_, dist_, ___}] := . . .

If in each of these we replace i_Symbol with i_Symbol | i : _Symbol[___] and evaluate the cells we find that the problem is resolved:

ParallelTable[a[1], {a[1], 0, 10}]    (* no error message *)

{0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10}

In light of this I think it was an accident to restrict the iterators to bare Symbols, and I propose tagging this as a bug.

Mr.Wizard
  • 271,378
  • 34
  • 587
  • 1,371
  • This is definitely an unwanted feature, as generating arbitrary-dimensional iterators and other such variables is often done with constructs like c /@ Range[dim], which is much less cumbersome than working with lists of Symbol[...], Unique[], and such. – kirma Jun 24 '15 at 13:34
  • Very enlightening, thank you. I don't know how to report a bug. – anderstood Jun 24 '15 at 14:44
  • @anderstood You're welcome. I think under the Help menu you should find an item Give Feedback... that can be used to report bugs, among other things. Or try emailing support@wolfram.com. – Mr.Wizard Jun 25 '15 at 05:30
  • 1
    I sent them a message with Give Feedback. They answered today: I have filed a report with our developers on this issue with ParallelTable. This may allow them to address it in a future version of Mathematica. – anderstood Jun 26 '15 at 14:20
  • 1
    @anderstood Have you heard back from them on this one? The original code works just fine on v10.4. – Emilio Pisanty Mar 23 '16 at 14:42
  • @EmilioPisanty: No. Thank you for the update. – anderstood Mar 23 '16 at 16:35
5

This seems to just be a limitation of ParallelTable, can't comment whether that has a deeper reason due to parallelism or is just a simple oversight. I think it was not possible to use expressions like a[i] as e.g. iterators in older versions but in newer version that has been added as a feature to many functions, but obviously not ParallelTable (as of 10.0.2, still doesn't work in 10.1.0). That is somewhat inconsistent, and it might work with future versions, but for the moment you'd have to use a workaround as Kattern has suggested and use explicit symbols instead of the a[i] like e.g.:

asym[i_Integer] := Symbol["a" <> ToString[i]];
With[{
    iter = Sequence @@ Table[{asym[j], -1, 1}, {j, 1, 3}]
  },
  ParallelTable[$KernelID -> f[Table[asym[i], {i, 1, 3}]], iter]
]

alternatives to create symbol names would include:

asym[i_Integer] := asym[i] = Module[{a}, a];
asym[i_Integer] := asym[i] = Unique["a"];

each has there advantages and disadvantages, a common problem is that you have to manually cleanup as they create a bunch of symbols instead of just one as your original approach...

Albert Retey
  • 23,585
  • 60
  • 104