The graph displays the year-over-year number of citizen reports of human feces in the city. It certainly seems like it’s getting worse. In fact, the number of people defecating on the streets between 2001 and 2018 has increased by over 400%! This is confirmed by many news headlines reporting on the graph when it was first released. A few examples are:

Sure seems like a dismal outlook, almost a disaster fit for the Old Testament.

Or is it?

The data (number of reports of human feces) and the conclusion drawn from it (San Francisco is worse than ever) makes my measurement spidey sense tingle. I have a few questions about both the data and the report.

Does the data control for the City’s rollout of the 411 mobile app, which allows people to make reports from their phone?

Has the number of people with mobile phones from 2011 to the present increased?

Do we think the City’s media efforts to familiarize people with 411, the vehicle for reporting poop, could contribute to the increase?

The media loves to report on the poop map and poop graph as proof of San Francisco’s decline. Would extensive media coverage contribute to citizen awareness that it can be reported, therefore resulting in an increase in reports?

Is it human poop? (I know the answer to this: not always. Animal poop and human poop reports are logged and tagged together in City databases.)

Does the data control for multiple reports of the same pile? 911 stats have this problem; 300 calls about a car accident doesn’t mean there were 300 car accidents.

Knowing that a measurement and subsequent analysis starts with a well-formed question, we have to ask: are we measuring the wrong thing here?

I think we are!

Perhaps a better question we can answer with this data is: what are the contributing factors that may show a rise in feces reports?

A more honest news headline might read something like this: Mobile app, outreach efforts leads to an increase in citizens reporting quality of life issues

Here’s another take on the same data: