OpenStudio has developed a cloud based platform to solve this very problem. Two recent papers were presented at ASHRAE:
Macumber, 2014, A graphical tool for cloud-based building energy simulation
Long, 2014, Scaling building energy modeling horizontally in the cloud with openstudio
The calibration workflow is Measure based, meaning almost any OpenStudio method is available for calibration purposes (ex, swapping out full HVAC systems, changing geometry, or simple idf parameter changes). A good Measure resource is https://bcl.nrel.gov/
The OpenStudio calibration / optimization workflow accepts timeseries and monthly utility data. Objective functions can also be put into groups for multi-objective problems.
The suggested workflow is to first sample the variable space with one of the sampling algorithms available on the server (ex, LHS). Remove any non-impactful variables and choose appropriate bounds for the impactful variables. Run a calibration using a gradient based method (ex, Optim) for all continuous variables, a hybrid genetic / gradient based method (Rgenound) or choose a multi-objective algorithm such as NSGA2
Computing time is a function of base simulation time and the number of variables that are selected for calibration. All the available algorithms are parallelized meaning gradient calcs are done in parallel for Optim and Rgenoud and all population calcs for NSGA2 are run utilizing all available cores. Costs are always changing but right now you can get a 32 core box for $1.68/hr with the capability to add more boxes for larger problems.
@Amir Roth you tag "ASHRAE Guideline 14", can you provide a link or a description of this in your question? I am not familiar with it.
ASHRAE Guideline 14 (https://gaia.lbl.gov/people/ryin/publ...guideline14-2002Measurement%20of%20Energy%20and%20Demand%20Saving%20.pdf) is a reference protocol for measuring energy savings. One of the options is calibrated simulation and there are requirements for how close the simulation has to match measured data to qualify (see table 5-2 in the document). IPMVP (http://www.nrel.gov/docs/fy02osti/315...) option D is similar.
Take a look here too Neal: https://unmethours.com/question/15/wh...
I don't have proper time to write a detailed answer but to anyone who's feeling like doing, I'd include ORNL Autotune, Hydro Quebec ExcalibBEM, GenOPT
@BrianLBall : these two comments actually constitute an answer, not comments. can you post an answer instead?