11

I use mathematica on a computer with linux operating system. The computer has 2 cpus and each cpu has 4 cores, so there are totally 8 cores available.

Now I got confused with whether the evaluation of Eigenvalues is parallelized or not. I present two kinds of codes below

one use ParallelDo

ParallelDo[
    Eigenvalues[# + 
        ConjugateTranspose[#] &[Table[RandomReal[], {i, 5000}, {j,5000}]]
    ];
,{4}]//AbsoluteTiming

the time it takes is 135.12254 second. And use Top command in linux during the evaluation, it shows like below:

enter image description here

It show that 4 cores are trying their best, that's quite reasonable.

the other use Do only

Do[Eigenvalues[# + ConjugateTranspose[#] &[Table[RandomReal[], {i, 5000}, {j,5000}]]];,{4}]//AbsoluteTiming

this time it takes 109.004395 second. It evaluates even faster!!. And use Top, It shows like below:

enter image description here

Almost 800%, it means that this time, the evaluation uses all the cores? I don't know why it shows one 800% instead of eight 100%s. What is the difference?

If Eigenvalues really evaluates in a parallelized way automatically, why Parallelize[Eigenvalues[matrix]] gives error message saying that " Eigenvalues can't be parallized;proceeding with sequential evaluation"??

J. M.'s missing motivation
  • 124,525
  • 11
  • 401
  • 574
matheorem
  • 17,132
  • 8
  • 45
  • 115
  • 2
    Hit H in top, that should show all user threads – ssch Jun 01 '13 at 13:44
  • @ssch yeah! it shows all. I never hit H before. So 800% is the same as eight 100%s in essence, right? – matheorem Jun 01 '13 at 13:52
  • 3
    @matheorem Because Eigenvalues doesn't use Mathematica's parallelization. Parallelize can only deal with high level Mathematica constructs while Eigenvalues is implemented in a low level language for dense numerical matrices. Mathematica is just calling this low level multithreaeded implementation. – Szabolcs Jun 01 '13 at 14:58
  • 1
    The reason why top displays 100% four times is because there are four processes (namely subkernels) doing calculation using a single thread each. I guess the idea is that if you explicitly parallelize, you don't want Mathematica to interfere with your parallelization strategy. – celtschk Jun 01 '13 at 15:53
  • I voted to close this question. Although it's a reasonable thing to ask, it's not really a Mathematica issue, but rather a question of understanding the difference between thread-level and process-level parallelism, which can apply equally well to any software. If we take the question literally, I can't see how it can be answerable except with "yes, it does (using the MKL)". – Oleksandr R. Jun 01 '13 at 19:13
  • 5
    However it has a natural followup question: Which Mathematica functions use thread-level parallelism? – celtschk Jun 01 '13 at 19:18
  • @Szabolcs Thank you. I understand what you mean. And to my experience, if the matrix is small(e.g. 500x500), ParallelDo is always faster, But if the matrix is large enough, Do is alway faster and better (because it uses less memory) . celtschk add a good question. Is there a list of mathematica function use built-in multithread implementation. Because working with such multithread function, parallel it or not should be careful if we want to improve efficiency – matheorem Jun 02 '13 at 02:32
  • Closely related: Partial Differential Equation in Parallel – Jens Feb 17 '16 at 15:56

2 Answers2

11

Eigenvalues (and some other builtin functions) may internally use a very efficient multithreaded implementation (depending on what sort of matrix it's applied to, i.e. sparse or dense, exact or inexact, etc.)

It is important to understand that this does not use Mathematica's parallelization framework and it is not user-controllable, apart from setting how many threads to use. You can read more about this here.

How many threads to use by these functions is controlled by the "MKLThreads" system option:

SystemOptions["MKLThreads"]

Update: In recent versions of Mathematica the option is SystemOptions["ParallelOptions" -> "MKLThreadNumber"].

This is set to the number of cores of your CPU on the main kernel, however, it is set to 1 on all subkernels. This explains the CPU usage patterns you see. 800% means that the process is using 8 cores.

Generally, when using a function that is already internally multithreaded, manual Parallelizeation will either not do anything or will slow things down.

Community
  • 1
Szabolcs
  • 234,956
  • 30
  • 623
  • 1,263
  • There are a few other controls available via environment variables. But these are mainly about CPU affinity, whether the threads are persistent across invocations, whether the workload is partitioned statically or dynamically, &c. Most users probably don't want or care to change these. – Oleksandr R. Jun 24 '14 at 08:26
  • @FredSimons Thank you, corrected! – Szabolcs Feb 17 '16 at 10:08
2

I don't think you are Parallelize-ing the Eigenvalues. What your code is doing is generating 4 5000X5000 random Hermitian matrix and find the eigenvalues. You can check with a smaller order (say 5X5), like

m = # + ConjugateTranspose[#] &[Table[RandomReal[], {i, 5}, {j, 5}]];
ParallelDo[Eigenvalues[m];, {4}] // AbsoluteTiming
Do[Eigenvalues[m];, {4}] // AbsoluteTiming
Table[Eigenvalues[m], {4}] // TableForm

I define the matrix in m to make sure ParallelDo and Do are using the same matrix. For my laptop execution time for ParallelDo and Do are 0.011845 and 0.000189 (Do wins again). Last Table command will show you what exactly is happening. You can check each loop produce a complete set of eigenvalues. It is not like all the loops together produce the complete set.

Now regarding your query - why the parallelization takes more time than series execution - well, I don't know that well. Only thing I can say it is the effect of interdependence of kernels which sometimes also depends on how you set your loops. From my experience I would suggest that it is always good to do a test drive with both series and parallel to see what suits better with your code.

Sumit
  • 15,912
  • 2
  • 31
  • 73