Question-and-Answer Resource for the Building Energy Modeling Community
Get started with the Help page
Ask Your Question
7

Regression testing scripts

asked 2015-04-14 21:35:05 -0500

Chandan Sharma gravatar image

updated 2015-07-11 17:26:15 -0500

I need to run full regression test suite using compiled EnergyPlus builds. It seems EnergyPlusDevSupport repository had scripts (RunComparison.py etc) needed for running regression tests but that repository doesn't exist anymore? Or maybe its contents moved to some other repository. CMakeLists.txt has BUILDTESTING and ENABLEREGRESSION_TESTING set to ON but needs following details which I am struggling to provide.

set( REGRESSION_BASELINE_SHA "" CACHE STRING "SHA of baseline comparison" )``
set( COMMIT_SHA "" CACHE STRING "Commit sha of this build" )
set( REGRESSION_SCRIPT_PATH "" CACHE PATH "Path to regression scripts" )
set( REGRESSION_BASELINE_PATH "" CACHE PATH "Path to energyplus baseline folder to compare against" )

Any help is appreciated. Thanks.

edit retag flag offensive close merge delete

2 Answers

Sort by ยป oldest newest most voted
6

answered 2015-07-24 13:53:58 -0500

lefticus gravatar image

COMMIT_SHA and REGRESSION_BASELINE_SHA are mostly not necessary - you can set them to anything you want to, for the purposes of running the regression tests locally.

  1. Set up your baseline
    1. Compile the version of EnergyPlus that you want to build against (baseline)
    2. Run the unit tests. This generates the necessary output
  2. Clone the tools necessary for running the regression from https://github.com/NREL/EnergyPlusDev...
  3. Set up the version of EnergyPlus you are testing
    1. Be sure to configure in a directory separate from the one you build your baseline in
    2. Set REGRESSION_SCRIPT_PATHto the folder you cloned https://github.com/NREL/EnergyPlusDev... into
    3. Set REGRESSION_BASELINE_PATH to the build folder you compiled the 'baseline' in
    4. Set ENABLE_REGRESSION_TESTING to TRUE
    5. Compile as normal
  4. Run regressions
    1. At this point make test should run all tests including the regression comparisons
    2. Or for parallelization ctest -j<numjobs>
edit flag offensive delete link more

Comments

Thanks. The GitHub link of EnergyPlusDevSupport repository is saying 404 Page not found. Am I missing something here?

Chandan Sharma gravatar imageChandan Sharma ( 2015-07-28 08:07:41 -0500 )edit

Oh Snap! The repo requires authentication. @Edwin can we do anything about that?

Kyle Benne gravatar imageKyle Benne ( 2015-07-28 11:41:50 -0500 )edit

That repo is where defect files are maintained which could contain proprietary info. We can't open it up. We can, however, move the test scripts. They could even go right in NREL/EnergyPlus as far as I am concerned. @lefticus what will this affect on the CI end of things?

Edwin gravatar imageEdwin ( 2015-07-28 12:00:48 -0500 )edit

I did provide the scripts via private email, so no hurry on moving these items.

rraustad gravatar imagerraustad ( 2015-07-28 12:34:36 -0500 )edit

Thank you all for the help.

Chandan Sharma gravatar imageChandan Sharma ( 2015-08-24 20:28:07 -0500 )edit
5

answered 2015-07-24 12:08:35 -0500

updated 2015-07-24 12:09:55 -0500

In CMake there is a check box for ENABLE REGRESSION TESTING. Also there is still the local machine regression testing as we performed in the past and still perform to test for differences in the results (csv diff's).

edit flag offensive delete link more

Comments

Local machine regression scripts are what I was actually looking for at that time. Thought they were also on the StarTeam and probably moved to GitHub. Where can find them?

Chandan Sharma gravatar imageChandan Sharma ( 2015-07-28 08:07:43 -0500 )edit

These are now stored in a private archive repository.

rraustad gravatar imagerraustad ( 2015-07-28 09:49:53 -0500 )edit

Thank you Rich.

Chandan Sharma gravatar imageChandan Sharma ( 2015-07-28 10:18:17 -0500 )edit

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

 



Question Tools

1 follower

Stats

Asked: 2015-04-14 21:35:05 -0500

Seen: 246 times

Last updated: Jul 24 '15