Question-and-Answer Resource for the Building Energy Modeling Community
Get started with the Help page
Ask Your Question
9

Time series visualization tools?

asked 2017-10-02 22:38:37 -0500

updated 2017-10-03 16:53:39 -0500

Any recommendations out there for software that helps with time series data processing and visualization? For example it is common to want to compare energy model outputs, BAS trends, and data loggers. I have made plenty of excel scatter plots over the years, but am looking for something that is faster, more scalable, etc. DVview looks promising, but is there any documentation? What happens if a data set has trends at different time intervals or gaps in data?

Any advice on tools and data wrangling tremendously appreciated.

edit retag flag offensive close merge delete

6 Answers

Sort by » oldest newest most voted
13

answered 2017-10-03 17:30:29 -0500

@Eric Ringold mentioned Python and Pandas already, but I thought I'd add my two cents as I've been a very heavy user of Python/Pandas for years (and a casual source code developer for Pandas), which I used for many and various things including looking at energy model output.

The following is my own opinion, and I'm heavily biased towards Python.


Why Python?

I'd go 100% with Python and Pandas personally. Pandas (which was purposefully built to handle time series) offers the same concept as R dataframes (R data.table package if I'm not mistaken). It's amazing, makes tasks like up/downsampling and filtering really easy or filling gaps (DataFrame.resample, DataFrame.fillna, DataFrame.interpolate among other methods), and it's fast (most core functions are vectorized and written in Cython). I almost never use Excel anymore...

It's true that R has, still today, more statistical support. But it doesn't mean that Python doesn't: most functions 99% of us will ever need are already in the statsmodel package or another like it.

On the other hand, Python is a more versatile and broad spectrum language than R (meaning there are more well-developed and maintained packages for extremely varied things, so it's usually easier to do something non-statistical with Python), so if you have to learn something now, go with Python.

For example, say you wish to find a way to use scripting to change an EnergyPlus input file. I doubt you'll find a package in R to do that (could be wrong...). But in Python you'll find eppy!


Want to get Started?

If you have never used Python before, and especially if you're on Windows (*Nix users have Python installed by default), perhaps one of the simplest way to get started is to install an Anaconda Distribution which will make it real easy to install a Python distro with hundreds of scientific packages and a package manager. The (smart) alternative would be to use virtualenv, but I wouldn't start by this: if it looks complicated to you, just leave it for your next machine.

Then, to learn some decent amount of Python, I'd start by taking an online interactive tutorial at least (plenty out there, learnpython.org for example), and perhaps and online course, such as the free University of Michigan Programming for Everybody on Coursera. I'm quoting this specific one since I have taken it quickly when I wanted to recommend a course for colleagues, and it's a very good course (perhaps too basic on some aspects if you have extensive programming experience in other languages, I can't really tell since I took it while already experienced in Python).

Then look into Pandas, and use Jupyter notebook for exploratory work.

Other packages worth looking at: I also really like Seaborn for plotting, and if I want to create interactive graphs that I can share with anyone, I tend to use Plotly ... (more)

edit flag offensive delete link more

Comments

1

Great answer. More like 75 cents.

ericringold's avatar ericringold  ( 2017-10-03 18:15:29 -0500 )edit

For Python plotting, this presentation should help you find the solution you're looking for: Jake VanderPlas The Python Visualization Landscape PyCon 2017

As for learning, if you have some familiarity with another language, I recommend Dive Into Python 3, which can be found here for free.

Matthew_Boyd's avatar Matthew_Boyd  ( 2017-10-04 13:20:45 -0500 )edit

This Udemy course is a great resource for python/pandas/matplotlib/plotly/seaborn. You can usually grab the course for $15 and it comes complete with Jupyter notebook lectures and exercises.

Lyle K's avatar Lyle K  ( 2017-10-04 15:17:46 -0500 )edit
5

answered 2017-10-03 03:46:24 -0500

I mostly use R programming language (see open source CRAN website for download): It is easy to read in csv output from EnergyPlus and deal with different time intervals or missing values. R language is also quite flexible with options for producing charts. The only drawback I observed was longer read-in times of csv tables (if output is in detailed timesteps and files are very large) than with ResutlsViewer.

edit flag offensive delete link more

Comments

Thanks for the tip. I was wondering if R was used in this application, and suspected it had potential. The learning curve looks a little steep but it sounds like it could be worth it. When plotting with R, is there a way to have an interactive chart, where you can click to zoom in on data?

Dan McJacobson's avatar Dan McJacobson  ( 2017-10-03 08:24:54 -0500 )edit
1

I second R. It's got many packages for time series analysis. I recommend forecast and xts.

The learning curve is steep but worthwhile. dplyr is awsome.

Making charts in R is another skillset, with a (steeper?) learning curve. Most people opt for ggplot2. If you want interactive charts, use plotly, or export to d3.js

mdahlhausen's avatar mdahlhausen  ( 2017-10-03 12:02:33 -0500 )edit
1

Yes, I think, you can also create interactive charts in R (check R packages for it) For simple plots, it takes only a small learning effort: see built-in help for read.csv, read.table, and plot commands.

OS-user-AT's avatar OS-user-AT  ( 2017-10-03 13:08:03 -0500 )edit

I think the best way to learn R is actually just starting with the official documentation An Introduction to R. It provides the necessary foundation to build off of, which will be better than just straight off searching Stack Overflow for answers. Also, definitely use the RStudio programming environment.

Matthew_Boyd's avatar Matthew_Boyd  ( 2017-10-04 13:28:43 -0500 )edit
4

answered 2017-10-03 10:02:37 -0500

updated 2017-10-03 10:02:52 -0500

Here is some documentation for DView, but I don't think it answers your questions about time intervals or gaps.

You can also do a lot with python, especially the pandas and matplotlib libraries. Here are some examples, not limited to data visualization.

edit flag offensive delete link more

Comments

1

The original download and documentation have additional information about DView before it was recently open-sourced on GitHub.

MatthewSteen's avatar MatthewSteen  ( 2017-10-03 20:11:27 -0500 )edit
1

answered 2017-10-04 13:44:27 -0500

Another option for Windows is the Universal Translator.

This has been a joint venture between PG&E’s Pacific Energy Center (PEC) and Lawrence Berkeley National Laboratory (LBNL) with contributions from National Institute of Standards and Technology (NIST) and Quantum Energy Services & Technologies Inc. (QuEST). There were other numerous alpha testers who provided valuable testing information and feedback.

The UT3 development was a result of the "Development of Diagnostic and Measurement and Verification Tools for Commercial Buildings" initiative and was funded by the California Energy Commission Public Interest Energy Research (PIER) Program which supports public interest energy research and development that will help improve the quality of life in California by bringing environmentally safe, affordable, and reliable energy services and products to the marketplace.

The UT3 is software designed for the management and analysis of data from loggers and trend data from building management systems. The UT3 also has a software development kit (SDK) which describes how 3rd party developers can write their own UT3 extensions. Examples of extension types includes analysis modules and data importers.

edit flag offensive delete link more
0

answered 2017-10-03 15:14:03 -0500

yungahhh's avatar

Possible short term solution: I recently learned about XLSwing which is a python plug-in for Excel. I watched some youtube videos about this and learned you can use the Panda library to do some manipulation with the time series. It offers some plotting, zooming into the plot based on that demo. Learning curve seems less daunting if you're familiar with python and can probably adapt some of your existing Excel stuff.

edit flag offensive delete link more

Comments

I think you mean xlwings.

MatthewSteen's avatar MatthewSteen  ( 2017-10-03 16:06:44 -0500 )edit

This XL Swing? :)

Julien Marrec's avatar Julien Marrec  ( 2017-10-03 17:34:07 -0500 )edit
0

answered 2017-10-04 13:08:25 -0500

Matthew_Boyd's avatar

I second the R and Python/Jupyter answers, but if you're looking for a non-programming solution I recommend KST Plot. A little quirky, but it's an extremely powerful cross-platform time series data visualization software. It's like DView, but it can handle gaps and instantly zoom and scroll millions of points at a time (if you just plot the lines, don't actually plot the points). For documentation I recommend their YouTube channel.

edit flag offensive delete link more

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Careers

Question Tools

6 followers

Stats

Asked: 2017-10-02 22:38:37 -0500

Seen: 1,856 times

Last updated: Oct 04 '17