Regression testing scripts

asked 2015-04-14 21:35:05 -0500

Chandan Sharma

2422 ●1 ●2 ●9

updated 2015-07-11 17:26:15 -0500

__AmirRoth__

4436 ●5 ●17 http://bleedinggreenna...

I need to run full regression test suite using compiled EnergyPlus builds. It seems EnergyPlusDevSupport repository had scripts (RunComparison.py etc) needed for running regression tests but that repository doesn't exist anymore? Or maybe its contents moved to some other repository. CMakeLists.txt has BUILDTESTING and ENABLEREGRESSION_TESTING set to ON but needs following details which I am struggling to provide.

set( REGRESSION_BASELINE_SHA "" CACHE STRING "SHA of baseline comparison" )``
set( COMMIT_SHA "" CACHE STRING "Commit sha of this build" )
set( REGRESSION_SCRIPT_PATH "" CACHE PATH "Path to regression scripts" )
set( REGRESSION_BASELINE_PATH "" CACHE PATH "Path to energyplus baseline folder to compare against" )

Any help is appreciated. Thanks.

edit retag flag offensive close merge delete

add a comment

6

answered 2015-07-24 13:53:58 -0500

lefticus
61 ●1 ●1

COMMIT_SHA and REGRESSION_BASELINE_SHA are mostly not necessary - you can set them to anything you want to, for the purposes of running the regression tests locally.

Set up your baseline
1. Compile the version of EnergyPlus that you want to build against (baseline)
2. Run the unit tests. This generates the necessary output
Clone the tools necessary for running the regression from https://github.com/NREL/EnergyPlusDev...
Set up the version of EnergyPlus you are testing
1. Be sure to configure in a directory separate from the one you build your baseline in
2. Set REGRESSION_SCRIPT_PATHto the folder you cloned https://github.com/NREL/EnergyPlusDev... into
3. Set REGRESSION_BASELINE_PATH to the build folder you compiled the 'baseline' in
4. Set ENABLE_REGRESSION_TESTING to TRUE
5. Compile as normal
Run regressions
1. At this point make test should run all tests including the regression comparisons
2. Or for parallelization ctest -j<numjobs>

edit flag offensive delete link

Comments

Thanks. The GitHub link of EnergyPlusDevSupport repository is saying 404 Page not found. Am I missing something here?

Chandan Sharma ( 2015-07-28 08:07:41 -0500 )edit

Oh Snap! The repo requires authentication. @Edwin can we do anything about that?

Kyle Benne ( 2015-07-28 11:41:50 -0500 )edit

That repo is where defect files are maintained which could contain proprietary info. We can't open it up. We can, however, move the test scripts. They could even go right in NREL/EnergyPlus as far as I am concerned. @lefticus what will this affect on the CI end of things?

Edwin ( 2015-07-28 12:00:48 -0500 )edit

I did provide the scripts via private email, so no hurry on moving these items.

rraustad ( 2015-07-28 12:34:36 -0500 )edit

Thank you all for the help.

Chandan Sharma ( 2015-08-24 20:28:07 -0500 )edit

add a comment

5

answered 2015-07-24 12:08:35 -0500

rraustad

13599 ●72 ●12 http://www.fsec.ucf.edu/

updated 2015-07-24 12:09:55 -0500

In CMake there is a check box for ENABLE REGRESSION TESTING. Also there is still the local machine regression testing as we performed in the past and still perform to test for differences in the results (csv diff's).

edit flag offensive delete link

Comments

Local machine regression scripts are what I was actually looking for at that time. Thought they were also on the StarTeam and probably moved to GitHub. Where can find them?

Chandan Sharma ( 2015-07-28 08:07:43 -0500 )edit

These are now stored in a private archive repository.

rraustad ( 2015-07-28 09:49:53 -0500 )edit

Thank you Rich.

Chandan Sharma ( 2015-07-28 10:18:17 -0500 )edit

add a comment