When calibrating an energy model, what are the typical metrics to show proper calibration.

answered 2014-09-04 03:15:04 -0600

Julien Marrec

29716 ●94 ●76 http://www.effibem.com/

updated 2014-09-05 02:44:48 -0600

The most important ones are:

(Normalized) Mean Bias Error, NMBE/MBE. This assess whether you globally over or under-predict the consumption.
Coefficient of Variation of the root mean square error, CV(RMSE). This basically assess how close you are to the individual data points (such as monthly utility bills).

The materials you want to read are:

principally the ASHRAE Guideline 14 "Measurement of Energy And Demand Savings" (2002). It details how to calculate such metrics (and quotes another one, CVSTD, for weather independent cases). For example, if you calibrate monthly (12 utility bills), CV(RMSE) shouldn't exceed 15% and NMBE shouldn't exceed 5%. Hourly: CV(RMSE) < 30% and NMBE < 10%
Guideline 14 is one of the basis behind the International Performance Measurevement and Verification Protocol (IPMVP) which I suggest you also read (volume 1, especially appendix B)

I haven't tried it but I know that OpenStudio lets you enter utility bill data and then calculates those metrics for you after your run.

edit flag offensive delete link

Comments

@Matt Larson: if you're satisfied by an answer, please remember to mark it as correct, so that your question is flagged as resolved. See: http://meta.stackexchange.com/questio... If it is not satisfying, don't hesitate to add comments to an answer to give clues for anyone else who might have a go at it. Thanks!

Julien Marrec ( 2014-09-11 09:17:00 -0600 )edit

Thank you for the detailed answer. I have used the utility bill feature in OpenStudio and that does provide a good graphic representation on your monthly calibration, along with your NMBE and CvRMSE compared to ASHRAE Guideline 14.

Matt Larson ( 2014-09-11 10:20:52 -0600 )edit

Btw, obviously that's only going to work if you're using an AMY weather file. Otherwise, there's a - somewhat poor except for simple buildings - workaround which consists in doing a multiple regression analysis to weather normalize your bills (dry bulb only), and calculating NMBE&CV(RMSE) based on weather normalized bills and outputs from TMY3 simulation.

This is notably used for New York (NYSERDA MPP) and New Jersey (P4P) multifamily programs, and Etracker by John Kelly Kissock (based on ASHRAE RP1050) is being used for the regression (I'd rather do my own regression but that's just me)

Julien Marrec ( 2014-09-11 10:34:26 -0600 )edit

add a comment