Question-and-Answer Resource for the Building Energy Modeling Community
Get started with the Help page
Ask Your Question
7

Regression testing scripts

asked 2015-04-14 21:35:05 -0600

Chandan Sharma's avatar

updated 2015-07-11 17:26:15 -0600

I need to run full regression test suite using compiled EnergyPlus builds. It seems EnergyPlusDevSupport repository had scripts (RunComparison.py etc) needed for running regression tests but that repository doesn't exist anymore? Or maybe its contents moved to some other repository. CMakeLists.txt has BUILDTESTING and ENABLEREGRESSION_TESTING set to ON but needs following details which I am struggling to provide.

set( REGRESSION_BASELINE_SHA "" CACHE STRING "SHA of baseline comparison" )``
set( COMMIT_SHA "" CACHE STRING "Commit sha of this build" )
set( REGRESSION_SCRIPT_PATH "" CACHE PATH "Path to regression scripts" )
set( REGRESSION_BASELINE_PATH "" CACHE PATH "Path to energyplus baseline folder to compare against" )

Any help is appreciated. Thanks.

edit retag flag offensive close merge delete

2 Answers

Sort by ยป oldest newest most voted
6

answered 2015-07-24 13:53:58 -0600

lefticus's avatar

COMMIT_SHA and REGRESSION_BASELINE_SHA are mostly not necessary - you can set them to anything you want to, for the purposes of running the regression tests locally.

  1. Set up your baseline
    1. Compile the version of EnergyPlus that you want to build against (baseline)
    2. Run the unit tests. This generates the necessary output
  2. Clone the tools necessary for running the regression from https://github.com/NREL/EnergyPlusDev...
  3. Set up the version of EnergyPlus you are testing
    1. Be sure to configure in a directory separate from the one you build your baseline in
    2. Set REGRESSION_SCRIPT_PATHto the folder you cloned https://github.com/NREL/EnergyPlusDev... into
    3. Set REGRESSION_BASELINE_PATH to the build folder you compiled the 'baseline' in
    4. Set ENABLE_REGRESSION_TESTING to TRUE
    5. Compile as normal
  4. Run regressions
    1. At this point make test should run all tests including the regression comparisons
    2. Or for parallelization ctest -j<numjobs>
edit flag offensive delete link more

Comments

Thanks. The GitHub link of EnergyPlusDevSupport repository is saying 404 Page not found. Am I missing something here?

Chandan Sharma's avatar Chandan Sharma  ( 2015-07-28 08:07:41 -0600 )edit

Oh Snap! The repo requires authentication. @Edwin can we do anything about that?

Kyle Benne's avatar Kyle Benne  ( 2015-07-28 11:41:50 -0600 )edit

That repo is where defect files are maintained which could contain proprietary info. We can't open it up. We can, however, move the test scripts. They could even go right in NREL/EnergyPlus as far as I am concerned. @lefticus what will this affect on the CI end of things?

Edwin's avatar Edwin  ( 2015-07-28 12:00:48 -0600 )edit

I did provide the scripts via private email, so no hurry on moving these items.

rraustad's avatar rraustad  ( 2015-07-28 12:34:36 -0600 )edit

Thank you all for the help.

Chandan Sharma's avatar Chandan Sharma  ( 2015-08-24 20:28:07 -0600 )edit
5

answered 2015-07-24 12:08:35 -0600

updated 2015-07-24 12:09:55 -0600

In CMake there is a check box for ENABLE REGRESSION TESTING. Also there is still the local machine regression testing as we performed in the past and still perform to test for differences in the results (csv diff's).

edit flag offensive delete link more

Comments

Local machine regression scripts are what I was actually looking for at that time. Thought they were also on the StarTeam and probably moved to GitHub. Where can find them?

Chandan Sharma's avatar Chandan Sharma  ( 2015-07-28 08:07:43 -0600 )edit

These are now stored in a private archive repository.

rraustad's avatar rraustad  ( 2015-07-28 09:49:53 -0600 )edit

Thank you Rich.

Chandan Sharma's avatar Chandan Sharma  ( 2015-07-28 10:18:17 -0600 )edit

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Training Workshops

Careers

Question Tools

1 follower

Stats

Asked: 2015-04-14 21:35:05 -0600

Seen: 871 times

Last updated: Jul 24 '15