Why does summation over list elements are very slow compared to direct numbers?

Question

Here I will provide a simple example of what I mean. Using ParallelSum with direct numbers is very fast as here

     Module[{stp = 0.002}, 
      ParallelSum[stp^2 Sin[x y]^2, {x, -3., 3., stp}, {y, -3., 3., 
        stp}]] // AbsoluteTiming
{5.01413, 16.4759}

Now assume that I have a random list of {x, y} and would like to perform the ParallelSum form the list as follows: the random list is in the form

Module[{stp = 0.002}, 
 listF = Flatten[
    ParallelTable[{x, y}, {x, -3., 3., stp}, {y, -3., 3., stp}], 1];]

and then

Module[{stp = 0.002}, 
  Sum[stp^2 Sin[listF[[i, 1]] listF[[i, 2]]]^2, {i, 
    Length[listF]}]] // AbsoluteTiming
{15.2268, 16.4759}

as you can see it is 3 times slower now?!

Note: if I use ParallelSum in the last part instead of Sum it is even much more slower.

If you often need to optimize operations like this, you should dive into vectorized operations and packed arrays. They can make orders of magnitude difference, compared to parallelization which is less than linear in the nr of kernels (less because of overhead). — Marius Ladegård Meyer, Jun 25 '21 at 21:32
Absolutely! Leonid Shifrin's answer to this is a good place to start. The other answers are of course also informative. Any answer provided by @ciao is in my opinion very often examples of this practice. — Marius Ladegård Meyer, Jun 26 '21 at 11:51

score 1 · Answer 1 · answered Jun 25 '21 at 20:22

By using a combination of Map and Total we can speed up the sum quite a bit:

With[{stp = 0.002},
  listF = Tuples[Range[-3., 3., stp], 2];
  stp^2 * Total[Sin[#[[1]] * #[[2]]]^2 & /@ listF] // AbsoluteTiming]
(*    {2.03412, 16.4759}    *)

I would assume that Sum assumes that the argument might depend on the index i in a way that prevents certain optimizations. Map, on the other hand, does away with the index, which allows for speedier execution in general.

Why does summation over list elements are very slow compared to direct numbers?

1 Answers1