Why does Open Studio use the DayofWeek for the start date to determine whether it's a historical year or typical year weather file ?

asked 2017-02-03 19:02:40 -0600

Joe Huang
2772 ●5 ●14

updated 2017-05-29 09:29:39 -0600

(I've moved this post from EnergyPlus_Support to Unmet Hours because it deals with some inner workings of EnergyPlus, EPW weather format, and Open Studio.. Although the opening question asks about how Open Studio uses a certain feature in the EPW format to distinguish between a "typical year" and a historical year weather file, I believe the underlying problem rests in the EPW format and EnergyPlus)

I've just been told by a client that Open Studio checks the DayofWeek for the start date on the EPW to the DayofWeek calculated from the date itself to determine whether the weather file is of a historical year or a "typical year" (see EpwFile.cpp) and thus would I mind changing that parameter on my EPWs which right now always sets the start date as a Sunday (I think it's hardwired in the WeatherConverter since I don't see anything in the DEF that allows that to be changed).

I find this request more than a little bizarre, because (1) whether a file is run as a "real year"or a "typical year" is a simulation option and NOT a condition of the weather file! (2) even if we accept this logic of comparing the start date DayofWeek to what it is actually to tell whether the file is a historical file or a "typical year" file, this seems like a very convoluted and error-prone way since someone (me ?) would have to calculate or look up on the Web the actual day of week for Jan 1 of that weather file, put that into the EPW, just so EnergyPlus will know "aha, this is a historical year weather file!".

This step is also completely unnecessary because EnergyPlus (as well as any other decent simulation program) has a Julian Calendar in its code to determine the day of week for any day, so why do we have to input that manually once again just to "prove" that this is a historical year run?

Let's see how another program (DOE-2) addresses this problem, which I never thought was a problem until now... In DOE-2, one enters the RUN-PERIOD is its entirety, i.e., Jan 1, 1999 through Dec. 31, 1999. If it's a "typical year" weather file that you want to start on a Sunday, then you pick a year where that happens - problem solved, no need to modify the weather file, of all things.

edit retag flag offensive close merge delete

Comments

@Joe Huang, for what it's worth, this site is not only for OpenStudio. 1400+ of the 3400+ questions have to do with EnergyPlus.

__AmirRoth__ ( 2017-02-04 17:24:17 -0600 )edit

add a comment

3 Answers

Sort by » oldest newest most voted

answered 2017-02-03 20:54:30 -0600

macumber

12018 ●23 ●21 http://automaticmagic.com/

When loading an EPW file, OpenStudio checks the file to see if it is AMY or TMY. This check is performed by seeing if all of the dates listed are in order, that the presence or absence of a leap day is consistent with the actual year, and that the data in the header is consistent with the actual year. If the file passes these checks then it is assumed to be an AMY file and OpenStudio sets the simulation run period to that actual year, otherwise OpenStudio sets the run period to the start day of week option. Certain functions in OpenStudio (such as calibration) are only available when an actual year is specified. The user can override the simulation run period if they want to specify that a file represents a specific year, they can even do this to perform calibration using TMY data (which is probably a bad idea). Therefore, it is a pain that the year from the AMY file is not brought into OpenStudio automatically if there is inconsistent data in the header. However, the user can override this setting and provide the actual year.

As I understand it there is some work underway to clean up how dates are represented in EnergyPlus. It is my hope that effort will clarify which settings are important and which are not.

edit flag offensive delete link

Comments

I also want to point out that leap years are also commonly missing from AMY data, this is going to be a issue for anyone looking to use AMY data from 2016. I'd like to see EnergyPlus post some documentation about how it handles missing leap days in AMY files. For example, if the RunPeriod says it is 2016 but there is no 2/29 in the file what happens? Is 2/28 run twice? Does it fail? What happens if 2/29 is in the file but it is not a leap year?

macumber ( 2017-02-03 20:57:32 -0600 )edit

Thanks for your answer, but I think that you're way overthinking the problem, which I actually never thought was a problem until now. I found what you wrote rather puzzling. If Open Studio checks all those things in the EPW (consistent year, DOW, leap day, etc.), then why does it also have to check the DOW for January 1 in the EPW header ?? I mean, if the EPW passes the other three checks, then why does it need the fourth? Extreme vetting? Also keep in mind that the DOW in the EPW header is inserted by the WeatherConverrter, which I've heard first hand is hard-coded to spit out Sunday!

Joe Huang ( 2017-02-03 23:11:35 -0600 )edit

Ran out of space, but I'm also surprised by your contention that "leap years are also commonly missing from AMY data" ?? How can that be? If someone processes actual recorded data, it's bound to have the leap day unless someone goes to purposely delete it, which would make no sense to me. I've created over 150,000 historical weather files and all of them are complete years. The only times I have deleted Feb 29 is at the request of some customers because the extra day crashes their software (ClimateConsultant, SAMS, Helioscope), not the other way around.

Joe Huang ( 2017-02-03 23:19:43 -0600 )edit

Is the day of week in the EPW file anywhere else besides the header? As far as I know that is the only place that it is located. I suppose that if OpenStudio checks the consistent year and leap day in the data section then it could ignore the day of week in the header. But it still seems to me like files should have the correct day of the year in that field.

We were mostly seeing the missing leap days in weather files purchased from Weather Analytics in 2012. Then we had a few glorious years with no leap days :-) But now I am wondering if we will start to see the problem again.

macumber ( 2017-02-04 00:44:56 -0600 )edit

I don't think the weather file should contain extraneous non-weather-related information like day of week. If a simulation needs to know the day of week, then it's the responsibility of the simulation program and, for that matter, ANY simulation program of which I'm aware is fully capable to figure that out from the time stamp! If WeatherAnalytics produced historical weather data missing leap days, that just reveals poor data integrity and a suspect methodology on their part. I'd challenge you to look at other sources of historical weather data and show that this is a prevailing problem.

Joe Huang ( 2017-02-06 16:46:53 -0600 )edit

see more comments

answered 2017-02-03 19:06:24 -0600

Joe Huang
2772 ●5 ●14

updated 2017-02-04 05:38:34 -0600

Julien Marrec

29726 ●94 ●76 http://www.effibem.com/

I've been silent on this issue for the past week because I was busy at ASHRAE, not because I had lost interest. Now that I'm back home, I'd like to add some following comments about Line 8 of the epw format that to me is the source of the problem. Line 8 states:

DATA PERIODS, 1,1,Data,Sunday, 1/  1,12/31

If others are like me, you've probably completely ignored this line as superfluous or redundant documentation. However, its impact on any building energy simulation done using historical weather data, such as calibrating an existing building model to utility bills, will be quite significant since most buildings, especially commercial, have very different operational schedules and occupancies depending on the day of week. Therefore, it seems obvious to me that the current situation of always having the day of week beginning on a Sunday needs to be changed. I've been told that users have the option of specifying the starting day of week that would overwrite what's stated in the epw weather file, but that seems convoluted (since the epw overwrites the day of week calculated by EnergyPlus from the date stamp!) and dangerous since most users probably have no idea this is something they have to do whenever using an actual year weather file.

The OpenStudio tactic of comparing the Day of Week (DOW) on the epw (always Sunday till now) to what EnergyPlus calculates from the date stamp to decide which to use is even more flawed. I've written a small awk routine to calculate the DOW for Jan 1 from 1980 through 2030 (see attached). Note that out of those 50 years, there are 7 (1984, 1989, 1995, 2006, 2012, 2017, 2023) that do begin on a Sunday! Therefore, this comparison is not reliable. If one has a "typical year" weather file that happens to use January from those years, Open Studio would think that's a historical file and use the DOW as calculated by EnergyPlus from the date stamp. On the other hand, it will interpret all the historical files of the other 43 years as "typical year" data and use Sunday as the starting DOW, which would completely mess up any correlation with the actual DOW for that year.

If I were working with a blank slate, I would say the best thing to do with Line 8 is to get rid of it, or at least the DOW specification that now always appears as Sunday. However, I am very much aware of the need to maintain data compatibility and avoid version clash between different epws, so I would not advocate such a course of action, i.e., change the epw. A couple of solutions I've thought of, neither of which is very difficult, are:

(1) modify EnergyPlus to ignore Line 8 of the epw, and expand the RUN-PERIOD object to include the year. The slight disadvantage of this solution is that if users want ... (more)

edit flag offensive delete link

Comments

I know I'm going to regret getting involved, but here goes anyway. Line 8 is currently the only way to reliably figure out the full time of the data points in the EPW. Some readily available EPW files have "60" in the minutes column. If read in as-is, this puts the first data point at 2AM on 1/1 rather than 1AM where it is supposed to be. OpenStudio doesn't use the minutes column for this reason: a cross check is done on line 2514 in EpwFile.cpp

Jason DeGraw ( 2017-02-06 07:56:16 -0600 )edit

The character limit for comments is pretty low. Continuing from my previous comment, there is a proposal to implement something like (1) in EnergyPlus. It would add an optional year field to the run period object, and for a requested start day of the week would figure out a year that works (using a shorter version of your table). If the (optional) start year and start day of the week don't agree, the current plan is to error out and ask for corrections in the IDF (which to me is better than asking for fixes in the EPW).

Jason DeGraw ( 2017-02-06 08:10:34 -0600 )edit

I would not rely on Line 8 for much of anything, because (1) it is a product of the WeatherConverter (WC), not something that's in the original weather file, (2) its contents are not even addressable by a user of the WC but written automatically by the WC. I just did an experiment where I chopped out January in a sample source weather file (CSV) and then converted that to EPW. What I got on Line 8 was "DATA PERIODS,1,1,Data,Sunday, 2/ 1,12/31" In other words, Line 8 always states the DOW of the beginning day to be a Sunday for any weather file that's being processed.

Joe Huang ( 2017-02-06 17:04:42 -0600 )edit

Same problem for me with the character limit for comments :-) :-) If a weather file has a suspect time stamp, e.g., 60 in the minutes column, that should be corrected or addressed in processing that weather file into an EPW, rather than be passed through to the EPW. I have never seen any such problem in the weather files I've processed because the time stamp is always complete and an output from the internal clock of the datalogger. Maybe you're not looking at real data but derivative data with poor post-processing.

Joe Huang ( 2017-02-06 17:13:06 -0600 )edit

Hopefully, WeatherConverter handles "Number of Records per hour" better, that's what OpenStudio uses to set minutes. I had planned to improve OpenStudio EPW handling, but that funding got cut. I'm now hoping to get a crack at it after modification (1) is in E+. Unfortunately, the files with bad time stamp are EPWs that are in the wild, e.g. this one. So your last sentence is pretty accurate.

Jason DeGraw ( 2017-02-08 09:50:16 -0600 )edit

see more comments

answered 2017-03-10 17:38:22 -0600

mdahlhausen

9499 ●19 ●11 http://openstudio.net/

Note for people trying to simulate with leap year data:

You need to make sure your .epw file has 8784 data fields, the leap year option is set to 'Yes' in the HOLIDAYS/DAYLIGHT SAVINGS header:

HOLIDAYS/DAYLIGHT SAVINGS,Yes,0,0,0

AND make sure you have the correct start day in the DATA PERIODS header. E.g., for 2016:

DATA PERIODS,1,1,Data,Friday,1/1,12/31

edit flag offensive delete link

Comments

This brouhaha about Leap Days is getting curiouser and curiouser, as Alice in Wonderland used to say :-) So EnergyPlus is defining Leap Days as a Holiday/Daylight Savings option?? That's totally bizarre. The Leap Day is an astronomical phenomenon because a year is roughly 365.25 days long, regardless of your religion, lifestyle, or simulation program. I can't help but be amused that a year or so ago I mentioned the problem of Leap Days in eQUEST, and got the unhelpful suggestion "use EnergyPlus!". Well, now it turns out that EnergyPlus is even more arcane in dealing with Leap Days...

Joe Huang ( 2017-03-13 14:01:42 -0600 )edit

add a comment

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Why does Open Studio use the DayofWeek for the start date to determine whether it's a historical year or typical year weather file ?

Comments

3 Answers

Comments

Comments

Comments

Your Answer

Training Workshops

Careers

Question Tools

Stats

Why does Open Studio use the DayofWeek for the start date to determine whether it's a historical year or typical year weather file ? edit

Comments

3 Answers

Comments

Comments

Comments

Your Answer

Training Workshops

Careers

Question Tools

Stats

Why does Open Studio use the DayofWeek for the start date to determine whether it's a historical year or typical year weather file ?