Regression testing scripts

7

asked 10 years ago

Chandan Sharma

2442 ●1 ●2 ●9

updated 9 years ago

__AmirRoth__

4446 ●5 ●17 http://bleedinggreenna...

I need to run full regression test suite using compiled EnergyPlus builds. It seems EnergyPlusDevSupport repository had scripts (RunComparison.py etc) needed for running regression tests but that repository doesn't exist anymore? Or maybe its contents moved to some other repository. CMakeLists.txt has BUILDTESTING and ENABLEREGRESSION_TESTING set to ON but needs following details which I am struggling to provide.

set( REGRESSION_BASELINE_SHA "" CACHE STRING "SHA of baseline comparison" )``
set( COMMIT_SHA "" CACHE STRING "Commit sha of this build" )
set( REGRESSION_SCRIPT_PATH "" CACHE PATH "Path to regression scripts" )
set( REGRESSION_BASELINE_PATH "" CACHE PATH "Path to energyplus baseline folder to compare against" )

Any help is appreciated. Thanks.

add a comment

6

answered 9 years ago

lefticus
61 ●1 ●1

COMMIT_SHA and REGRESSION_BASELINE_SHA are mostly not necessary - you can set them to anything you want to, for the purposes of running the regression tests locally.

Set up your baseline
1. Compile the version of EnergyPlus that you want to build against (baseline)
2. Run the unit tests. This generates the necessary output
Clone the tools necessary for running the regression from https://github.com/NREL/EnergyPlusDev...
Set up the version of EnergyPlus you are testing
1. Be sure to configure in a directory separate from the one you build your baseline in
2. Set REGRESSION_SCRIPT_PATHto the folder you cloned https://github.com/NREL/EnergyPlusDev... into
3. Set REGRESSION_BASELINE_PATH to the build folder you compiled the 'baseline' in
4. Set ENABLE_REGRESSION_TESTING to TRUE
5. Compile as normal
Run regressions
1. At this point make test should run all tests including the regression comparisons
2. Or for parallelization ctest -j<numjobs>

link

Comments

Thanks. The GitHub link of EnergyPlusDevSupport repository is saying 404 Page not found. Am I missing something here?

Chandan Sharma ( 9 years ago )

Oh Snap! The repo requires authentication. @Edwin can we do anything about that?

Kyle Benne ( 9 years ago )

That repo is where defect files are maintained which could contain proprietary info. We can't open it up. We can, however, move the test scripts. They could even go right in NREL/EnergyPlus as far as I am concerned. @lefticus what will this affect on the CI end of things?

Edwin ( 9 years ago )

I did provide the scripts via private email, so no hurry on moving these items.

rraustad ( 9 years ago )

Thank you all for the help.

Chandan Sharma ( 9 years ago )

add a comment

5

answered 9 years ago

rraustad

13924 ●81 ●13 http://www.fsec.ucf.edu/

updated 9 years ago

In CMake there is a check box for ENABLE REGRESSION TESTING. Also there is still the local machine regression testing as we performed in the past and still perform to test for differences in the results (csv diff's).

link

Comments

Local machine regression scripts are what I was actually looking for at that time. Thought they were also on the StarTeam and probably moved to GitHub. Where can find them?

Chandan Sharma ( 9 years ago )

These are now stored in a private archive repository.

rraustad ( 9 years ago )

Thank you Rich.

Chandan Sharma ( 9 years ago )

add a comment