I've searched many questions about parallel processing here, and they are either specialized or over my head. I run the following parallel notebook on an 8 core EC2 instance. The average memory usage is ~40% of 32GB and the CPU utilization is ~50%. Using "CoarsestGrained" does not help. Why is it not maxing out the system capacity?
Remove["Global`*"];
LaunchKernels[];
mccount = 10^6;
cyclesperday = RandomVariate[LogNormalDistribution[-0.9, 1.3], 635554];
samplemanycyclesper5years =
ParallelTable[
Total[RandomChoice[cyclesperday, 1826]], {100*mccount}];
cycledatatofit = Partition[samplemanycyclesper5years, 100];
samplecycledistributions =
ParallelTable[
EstimatedDistribution[cycledatatofit[[i]],
LogNormalDistribution[\[Mu], \[Sigma]]], {i, 1, mccount}];
samplecycledistributions[[1]]
LaunchKernels@8– Fortsaint Dec 10 '18 at 22:08