Big Data, Schmig Data. How ‘Bout Some Perspective?

Last night I taught the 3rd of a 3-session private workshop in a nonprofit organization. The initial conversation was from the nonprofit’s IT Director, Margo, who requested, “We’re upgrading to Excel 2013 and would like someone to come in and show us what’s new.”

In session #3 we pulled out the “Slicers w/Tables” feature of Excel 2013. However, Margo was elated that most of our time together was spent re-thinking how they use their data, develop their spreadsheets, and endless potential that flows from just that. The summary:

Maintain contiguous datasets and open a new world of possibilities in analysis.

We just have to be crafty in how we structure the contiguous data table.

We have to have forethought in the Why and What we want from our data.

In a short 5 frikken hours, we grew from small questions about a single formula, and eventually saw that if we restructure the layout, we don’t need the formula AND we can use pivot charts and slicers to give us insights that seemed impossible.

Now we’re in the thrilling world of dashboards and visualization, not just a measly SUMIF formula.

Now we can really make some stuff happen by easily looking for correlations in a pivot chart, comparing performance over longer periods of time, all with confidence in our accuracy.

Now we’re expanding Data Literacy.

Now we’re thinking big picture Data Management and Business Intelligence!

Baby, now we’re taming Cerberus!

BIG DATA, SCHMIG DATA

Contrast that against the mishegoss of articles about Big Data and the hand-wringing over the need for Data Scientists. I think Margo is more on the ball by getting training for her people who are directly responsible for forwarding the nonprofit’s mission.

Data Science is sexy, sure. Entry-level salaries are over $110K. Data Scientists are expected to hold advanced degrees, write code in multiple programming languages, do the analytics and present stellar PowerPoint presentations to Executives.

To what end? One example: gather publicly available data about people, develop models of their internet behavior, then predict/identify those whom the company is interested in, e.g.,

Close to buying a luxury car

Likely to make their credit card payments

Other Data Scientists can help

Predict the spread of diseases

Predict election results (Hello, Nate Silver)

Big data is even showing up in bra design: Engineering the Perfect Big-Data Bra

However, there are far more people who still have to handle day-to-day in-the-trenches critical data. They aren’t trying to predict Porsche purchases. No. No. They get in trouble when crap data and poor spreadsheet development causes them to:

Rent a van instead of a bus for summer campers and chaperones.

Not report someone’s completed professional education requirements to the state.

Process payroll inaccurate or late.

Submit an incomplete grant application.

Wreck a department’s cashflow.

Mix up hotel room assignments for 50 people who are packed in the lobby.

Ok. Ok. A Data Scientist and a department coordinator in a nonprofit have different roles and relate to data in enormously different ways. I don’t suggest that the Big Data and Data Scientist discussions are hogwash. I just think that there needs to be some perspective because the Data Science talk is really getting boring. A Data Scientist isn’t the one on the hook when 50 irritated people can’t get to their rooms and unpack.1

This talk is also frustrating when there’s urgency to dig up more complexity, store it in complicated data warehouses run by database administrators who have other priorities, thereby further disempowering the people who need daily and regular access to the data. Data Literacy and Data Stewardship should be discussed more. Employers should identify the people who are intellectually curious who can learn this whole data thing at a “good enough” level.

IT’S NOT THE LARGE THINGS THAT SEND A MAN TO THE MADHOUSE

Charles Bukowski says it best in his poem, “The Shoelace”



Here are 2 excerpts:

1.

It’s not the large things that send a man to the madhouse

death he’s ready for

or murder, incest, robbery,

fire, flood.

No it’s the continuing series of small tragedies

that send a man to a madhouse

not the death of his love, but a shoelace that snaps

with no time left.

The dread of life

is that swarm of trivialities

that can kill quicker than cancer

…

2.

with each broken shoelace

out of 100 broken shoelaces

one man, one woman, one thing

enters a madhouse …

MORE DATA LITERACY, LESS HYPE

Let’s have more conversations about empowering the people who are often unwitting data managers.

One other very brilliant effort at data literacy is MigraHack, founded by Claudia Núñez and Phuong Ly. The core concern was that public conversations about immigration aren’t backed up by data. Part of the problem is that journalists aren’t data people, and data people don’t often paint a story around cold numbers. So, MigraHack brings together journalists, data people and developers to explore immigration issues that tell accurate stories.

The journalists, the Volunteer Coordinator at a nonprofit, or the shift supervisor in a hotel … they represent Bukowski’s 100 shoelaces, whether you’ve got a rockstar Data Scientist, or not. Be like Margo, Claudia, and Phuong. Tend to the shoelaces before they break. That can be enough to tame Cerberus.

Hercules & Cerberus Photo Credit: katty_meo

Bukowski image: from QuotesTree.com

1. This actually happened in Portland, OR. A group of us had flown in from Chicago and all of our room assignments were screwed up. It was the result of crap data, data inputs and data controls, not lazy or inept people.↩