Optimal number of parallel EnergyPlus simulations on multi-core machine
I was using Python multiprocessing and subprocess.run() to run multiple EnergyPlus simulations at the same time. Suppose I have a dual CPU machine with each CPU has 4 cores and 8 logical processors. As a rule of thumb, should the optimal number of parallel runs/subprocess be the number of CPUs (2), cores (8), or logical processors (16)?
The EnergyPlus documentation Run EnergyPlus in Parallel says "To be time efficient, the number of parallel EnergyPlus runs should not be more than the number of CPUs on a computer", but I'm not sure how it applies to multi-core CPUs, nor can I find other clear answers elsewhere. Any suggestions are more than welcome.
EDIT: Following the advice from @Jason Glazer and @Julien Marrec, I did a simple test with two models, 24 runs each on another laptop with a 4-core CPU, 8 logical processors, and 32GB RAM. No other tasks were performed on the laptop during the test. Customized pre- and post-process are performed separately from parallel runs in my workflow so they won't consume CPU during the runs.
Here's the results (number of processes - total time/time per EnergyPlus run):
Model 1:
- 3 - 163.84s/20.48s
- 4 - 142.31s/23.71s
- 6 - 133.41s/33.35s
- 8 - 123.81s/41.27s
Model 2:
- 3 - 190.89s/23.86s
- 4 - 162.97s/27.16s
- 6 - 144.02s/36.00s
- 8 - 136.09s/45.36s
So it looks like it's OK to use maximum number of threads, but the gain is very small.
After reading Accoding to this answer on StackOverflow , it sounds like using the number of cores (optionally minus 1) is a safer bet for my workflow. Would be interesting to see workflow, especially when the number of runs reach thousands. Not sure if it is still the case for pure EnergyPlus workflow, i.e. using EP-Launch's group simulation functionality.
Thank you again for your answers.advice.
