Question-and-Answer Resource for the Building Energy Modeling Community
Get started with the Help page
Ask Your Question

When calibrating an energy model, what are the typical metrics to show proper calibration.

asked 2014-09-01 22:25:32 -0500

When calibrating to either monthly utility bills, meters, sub-meters, or other means, are there variance or percent error metrics that show proper calibration? What are some sources for this information?

edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted

answered 2014-09-04 03:15:04 -0500

updated 2014-09-05 02:44:48 -0500

The most important ones are:

  • (Normalized) Mean Bias Error, NMBE/MBE. This assess whether you globally over or under-predict the consumption.
  • Coefficient of Variation of the root mean square error, CV(RMSE). This basically assess how close you are to the individual data points (such as monthly utility bills).

The materials you want to read are:

  • principally the ASHRAE Guideline 14 "Measurement of Energy And Demand Savings" (2002). It details how to calculate such metrics (and quotes another one, CVSTD, for weather independent cases). For example, if you calibrate monthly (12 utility bills), CV(RMSE) shouldn't exceed 15% and NMBE shouldn't exceed 5%. Hourly: CV(RMSE) < 30% and NMBE < 10%
  • Guideline 14 is one of the basis behind the International Performance Measurevement and Verification Protocol (IPMVP) which I suggest you also read (volume 1, especially appendix B)

I haven't tried it but I know that OpenStudio lets you enter utility bill data and then calculates those metrics for you after your run.

edit flag offensive delete link more


@Matt Larson: if you're satisfied by an answer, please remember to mark it as correct, so that your question is flagged as resolved. See: If it is not satisfying, don't hesitate to add comments to an answer to give clues for anyone else who might have a go at it. Thanks!

Julien Marrec's avatar Julien Marrec  ( 2014-09-11 09:17:00 -0500 )edit

Thank you for the detailed answer. I have used the utility bill feature in OpenStudio and that does provide a good graphic representation on your monthly calibration, along with your NMBE and CvRMSE compared to ASHRAE Guideline 14.

Matt Larson's avatar Matt Larson  ( 2014-09-11 10:20:52 -0500 )edit

Btw, obviously that's only going to work if you're using an AMY weather file. Otherwise, there's a - somewhat poor except for simple buildings - workaround which consists in doing a multiple regression analysis to weather normalize your bills (dry bulb only), and calculating NMBE&CV(RMSE) based on weather normalized bills and outputs from TMY3 simulation.

This is notably used for New York (NYSERDA MPP) and New Jersey (P4P) multifamily programs, and Etracker by John Kelly Kissock (based on ASHRAE RP1050) is being used for the regression (I'd rather do my own regression but that's just me)

Julien Marrec's avatar Julien Marrec  ( 2014-09-11 10:34:26 -0500 )edit

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Training Workshops


Question Tools



Asked: 2014-09-01 22:25:32 -0500

Seen: 2,489 times

Last updated: Sep 05 '14