Using CartoDB Deep Insights to visualize and analize flights data for 1M routes from FlightStats

Last week a friend asked on Twitter how to extract some data from Airline On-Time Performance Reports in a Tableau public dashboard. I replied back with some tricks using Tabula, when the raw data or a workbook aren't provided, and turned in.

While the data was interesting, it lacked location reference, thus eliminating the possibility of representing it on a map, not to mention further geospatial analysis. But the idea of visualizing it lived on.

Getting the data

Friday night, destroyed after a hard week. I got home, had some dinner, and started poking at FlightStats Developer Center. I grabbed a RedBull to stay awake, and wrote a script to get the rating for every route in a dataset from OpenFlights.org.

I validated the visualization with a sample of flights departing from Barcelona, with a very simple line visualization: a color ramp corresponding to a 5 star scale (red for 1, green for 5).

Several hours, a couple of "Authorization failed. usage limits are exceeded" , and different API keys later I had the data.

Explore and analyze

Fast forward to this week. We've been working on some new and shiny stuff at CartoDB that has just been presented at MWC — CartoDB for Deep Insights. I had the data ready, so I uploaded it to CartoDB, and created my first Deep Insights dashboard within a couple of minutes. BOOM.

A quick look at the visualization and I can start uncovering stories.

We're continuously working to improve this, so any feedback is welcomed. Go ahead and play with the data.

You can check the code in GitHub.

Disclaimer: I'm not a Data Scientist, just some guy who knows how to put together a script in Ruby and happens to work at CartoDB. I really, really, really got inspired by this post to share the visualization. I usually tinker a lot but never end up sharing anything because I'm too critical with the results, so please be kind.