Time-based Visualization With Rose Diagrams

I love my job.

Not everything about it. Having my hours cut to 20/wk is hard on the finances. Working primarily in Excel is often enough to make me want to carve my eyes out with a fork. And I know that it’s likely that it’ll be gone in a year from now, since it’s a contract with The Army Corps.

Sure, there are downsides. But fundamentally, it’s a really good gig. My working conditions are ridiculously comfortable, my boss is a super nice guy, it’s enough to pay the bills, etc. I grew up in The Projects- yes, those Projects. Now, here I sit making enough to own a house and listening to whatever music I want. I feel pretty damn lucky.

With all that, one of the best parts about my job is simply one word: data. I’ve spent the better part of my adult life organizing and manipulating environmental data. Here, it’s basically the only thing I need to be doing.

But, there are dangers. The main one is that data is sometimes important, and loosing data is a very bad thing. We have data loggers in the Columbia, Willamette, Snake and other rivers in Oregon, Washington and Idaho. Some of these data loggers transmit data via cell or satellite phone, which we then pick up and manipulate. One of those loggers is causing us a problem. It periodically drops out, and we loose data. That’s a bad thing.

So, in my tinkering ways, I started setting out to find ways to make a system automating the announcement of failure. I won’t detail it all, but what I will detail is a continuation of my work with some rose diagrams that I cobbled together using Google Chart.

You see, one thing I want to see, not just for this, but for much of our data, is what time certain events occur. And not just the time of individual events. Sometimes, it’s nice to view the general tendency of events with respect to time. And what best way to visualize time then look at a clock? So I thought: “Hey, could I make a clock out of my rose diagrams?” Turns out it was pretty darn easy.

Twitter updates by time of day

The result is available for you to play with. Rather than use some of our data, I decided to take a more rapidly changing, but easily parsed data stream, and what can be more “rapidly changing” than the fickle bastards on Twitter!?

So, with a bit of Google App Engine work, and a small amount of abstracting of my azimuth-based rose diagrams, I came up with a simple program that parses the last 500 updates of any user and builds a rose diagram based on their update times (adjusted by the timezone in the users Twitter account.)

What we see is a rapid visualization of the relative percentage of update volume by a given user organized by hour. The updates at the right are from my Twitter account. The visualization gives a decent rapid view of the breakdown by morning, afternoon, evening, etc.

Updates in the AM hours

What if we don’t want to view a 24-hour clock? What if we want a regular old clockface to look at? Well, we have to separate updates by AM and PM, but after that, it’s easy. It turns out that the rose diagram code was more robust than I thought. Here, we have my update frequency in the AM hours.

Despite staying up until 1:30 am last night working on this, I’m awake enough to be still excited about it. It shows me that it’s relatively painless to create automatically generated rose diagrams using various “scales” to the rose. We’re certainly not limited to azimuths at 360°, or clocks at 24 hours. We can use them all, and even within the same page, very easily.

Given how easy it is to graph a data stream, it still surprises me that we in the sciences just hardly ever actually do it. That’s something I’m aggressively trying to fix.

As I said, it’s just a one-off, to see what’s easily possible. Now the real work begins. If you’re interested,  visit the site and throw in your Twitter name to see your update frequency. The Google Apps Code is available on my Github account. It’s not clean, and I likely won’t do much to clean it up because I’ll be taking the fundamental portions of this to use for other projects, but you’re welcome to play with it.

No Comment