Parallel processing of jobs with OpenStudio 2.1 Linux

asked 2017-06-02 09:27:10 -0500

Yunyang Ye
806 ●3 ●9 https://energyenvironm...

updated 2017-08-06 07:30:17 -0500

To run multiple models, I modify the contents in compact.osw by using Python, and then change the models by using the compact.osw. After that, I can run the new models. (''' cd path/to/Examples/compact_osw ./path/to/openstudio.exe run -w compact.osw ''') Now, I want to make the works parallel. Because I use OpenStudio 2.1 in Ubuntu and decide to do sampling methods locally, I do not want to use PAT to do the work. Does anyone have some experience about such work. Are any packages in Python, such as joblib.Parallel, pp, recommended to do the work?

edit retag flag offensive close merge delete

add a comment

2 Answers

Sort by » oldest newest most voted

answered 2017-06-02 10:50:05 -0500

shorowit

10891 ●180 ●9 https://www.nrel.gov/b...

updated 2017-06-02 10:52:20 -0500

You could use python's threading module to make multiple calls to the CLI in parallel. The subprocess module can be used for the actual call. (For ruby, the parallel gem coupled with a system call would work nicely.)

You'll also need to specify a different run_directory (and maybe others?) in the OSW so that the runs don't try to use the same directory -- or simply put the OSWs in different directories.

edit flag offensive delete link

Comments

Thanks. I will try to do the work by using the method that you mention.

Yunyang Ye ( 2017-06-02 11:47:36 -0500 )edit

@Yunyang Ye Did this work?

Determinant ( 2017-09-13 18:51:31 -0500 )edit

@Determinant I am working on that in a new supercomputer now. It seems work, and if I have a new progress, I will tell you.

Yunyang Ye ( 2017-09-28 15:37:06 -0500 )edit

@Yunyang Ye that's really great. Thanks for doing this. If they stop supporting the PAT on Linux this will be our only way to go. Please do keep me informed.

Determinant ( 2017-10-04 17:36:54 -0500 )edit

I have running code using Python's subprocess.call. It's great but the problem is that the directories in the OSW have to be absolute. The default directory used by the openstudio command doesn't work for some reason. Absolute paths make the results subdirectories messy. Any ideas?

Determinant ( 2018-08-27 15:12:38 -0500 )edit

see more comments

answered 2017-06-02 09:32:30 -0500

ljbrackney

3816 ●2 ●14 http://openstudio.net/

You need to provision your machine with OpenStudio Server and invoke the OpenStudio Meta CLI directly for your application. OpenStudio Server has an embedded instance of R to manage parallelization of algorithms. Learn more here.