Calibration for OS-HPXML: Utility Bill Data period ordering
We think we found a bug in the OS-HPXML-Calibration overlap check (hpxml.py, lines 441-453). It compares utility bill periods by list index, assuming the list is already in chronological order — but it never sorts the list or validates that it is sorted. If bills arrive out of order, it flags unrelated periods as overlapping when they don't.
We hit this on a real home. The error claimed two periods seven months apart overlapped ("2024-09-08 to 2024-10-08" vs. "2024-01-05 to 2024-02-08"), which made no sense once we checked the actual submitted data — those two periods aren't even adjacent. We traced it to the unsorted-list assumption in the index-based comparison.
(Side note, not a bug: the "off by one" end dates in the error message are intended behavior — the code shifts the end timestamp to midnight to make the range inclusive of the full final day, which is why the displayed date looks shifted.)
Suggested fix: sort the list by start date before running the overlap check, or validate it's sorted and raise a clearer error if not.
We've worked around it on our end — we fixed our database query to always return utility bills in sorted order, so this isn't blocking us. Flagging in case it's useful for OS-HPXML-Calibration more broadly.
Thanks @muhl, I moved this bug report over to https://github.com/NatLabRockies/Open.... You are welcome to report issues directly to that repo if you want.
Ah. Right, that makes more sense for where it is helpful to post. Thanks. Will do in the future.
For what it's worth, it looks like we were intentionally throwing an error for utility bills that are out of order (we have a test file to specifically demonstrate that an error is hit). I think the idea was to come back around later and sort on the fly, but it was never a high enough priority to implement. We figured most software would already have their utility bills sorted.