It’s an age-old NYC problem. It’s some time after 4PM and you step out to look for a cab. You giggle as you watch a tourist repeatedly try and hail cabs with the off-duty lights on, but then you realize the joke is on you too. Most of the cabs have off-duty lights on. Better luck next time.

Any New Yorker knows this is not just a myth. It’s rough to trying to find a cab around 4pm, and it’s a problem that New York has been grappling with for decades. And if its real, we should be able to see it in taxi data, right?

First, the bad news. The Taxi and Limousine Commission (TLC) is still way behind the curve on the Open Data movement in New York City. While we are starting to see some big steps forward by the NYPD over the last year (e.g. new stop and frisk data, new motor vehicle collision data), we are still not seeing much from the TLC. Taxi data can tell us so much- I’ve used it to show that different cabs are charging different tips when riders push the same buttons, to show how much we tip in each neighborhood, or even to show when rush hour is. But the data is only accessible by filing Freedom of Information Law requests. That’s not how Open Data should work, and I once again call on the TLC to release data for public analysis.

Now, the good news- Chris Whong FOILed the data and posted it online to the public, filling in where the government has let us down. I looked at the data from 2013 to see if there was a significant drop in trips taken around 4 or 5PM. The line in yellow below shows the average revenue (from base fares) per minute generated by fares for the entire fleet of cabs in New York City. Notice anything?

The dreaded shift change is staring at us quite clearly. It may actually be worse than you thought. It starts to decline at about 2:50pm and reaches a low point at 4:30pm. It’s back to normal at around 6:10pm. That’s over 3 hours or reduced service which is bad for riders. And the data shows that drivers lose about 3% of their overall revenue as a whole due to the shift change (assuming that the fares could be the line in blue) and there seems to be a dip of about 20% in cab rides taken at 4:30PM.

Now you may ask yourself why cab drivers choose to do a shift change at a time of the day that has some of the highest demand and thus the highest potential for income. One of the main answers is sort of strange. It turns out that it’s a good place to split revenue between the two cab drivers so that each gets roughly equal pay for a 12 hour shift. It’s all about fairness in some ways.

To explore this, I plotted out the fare revenue for the previous and next 12 hours at each time of day. When the two lines intersect, that means that the previous 12 hours has been equal to the next 12 hours and so that would be an equitable time to hand the keys over to a new driver.

Based on base fares, the optimal time is 5:08AM/PM, a bit later than cab drivers usually hand things over these days but don’t tell that to the morning shift guy.

So how do we fix this? Under the Bloomberg Administration, the city tried to combat this by adding a rush hour surcharge. The thought is that if you add more incentives to be on the road during this time, you get more cabs. So they raised the fare from 4PM-8PM by $1. According to the New York Times, David Yassky, the commission of the TLC in 2011, said “We believe that the 20 percent dip (in available taxis) would be even worse if it weren’t for the surcharge.”

But look what happens when you add the $1 charge to the base fare. You get the lines in blue instead of the ones in yellow:

The lines in blue cross farther to the right, showing that the more equitable handoff is now later at 5:22PM instead of 5:08PM. This makes sense. Most of rush hour is driven by the later shift driver, who is suddenly making more money during most of his or her shift. To keep it fair, the handoff would have to happen a little bit later so that the extra revenue generated by the surcharge gets more evenly distributed between late shift and early shift drivers. So in fact, the economic theory might suggest that the surcharge the City imposed might have backfired… incentivizing drivers to change keys later into rush hour instead of earlier to keep things fair. (Note that this is just a theory as to why it did not work very well- we don’t have the data to test it well. The TLC has not released it.)

Assuming that the “throw money at rush hour” trick may not work, what other options do we have? I have come up with two proposed solutions and a few other ideas that in combination may just take a dent out of the problem:

1) Change the surcharges to shift the equilibrium point earlier.

I’ve come up with a fare structure that does just that. Imagine you had a -1.50 surcharge, (a reverse surcharge) from 3PM - 3AM and a $2.50 surcharge from 3AM - 3PM. Now let’s have a look at the new optimal times in red:



The total amount of pay per 12 hour shift remain the same for both drivers as the current system, but the shift change would be optimal at about 3PM instead of 5PM. This would increase the overall revenue of drivers since they would be doing the shift change at a time of less demand, leading to more overall taxi drivers. Win/Win. Of course, this is sort of an extreme surcharge set up, but this idea of changing surcharges to move the equilibrium is a real option. The City seems to have gone in the opposite direction with this approach.

2) Incentivize fleet owners to charge different car rental fees for different shift times.

If fleet owners changed their rental fees on a sliding scale to more fairly match expected revenue to a shift time, you would see more flexibility in the shift change. Many drivers pay about $130 to rent a cab for a twelve hour shift. As it stands, that’s true for the 5AM - 5PM shift, or the 5PM to 5AM shift. But what if a new 2AM - 2PM shift was created where cabs cost only $100 to rent, while the 2PM - 2AM shift, which would certainly bring in more income for the driver due to extra demand, cost $160. Both drivers may end up making the same amount as they did before, the fleet owner would make the same amount as before, and more taxis would be on the road during rush hour. (This may actually end up making both drivers more, since the total 24 hour taxi revenue would be higher due to the fact that the shift change is happening during less busy hours).

In order to incentivize fleet owners to have a sliding scale, the City could create suggested cab rental rates based on time of day, and even reward garages who successfully use it to spread out the shift changes. This approach is superior to option one above because it defuses or spreads out the shift changes, as opposed to just shifting them to an earlier time.

3) Other (less realistic and less practical) ideas:

Make the price of taxis change dynamically throughout the day to help balance the shifts.



Convince the driver of a more profitable shift to pay the driver of the less profitable one.

Have drivers alternate shifts every week so they sometimes get more profitable ones, and sometimes less.

Conclusion

So can this really take a dent out of the famous NYC rush hour taxi problem? I really believe that a data driven approach like this can and should be explored by the TLC. And given that it will help put part of the missing 3% of taxi revenue back in the pockets of drivers, it seems like we might be able to pull this off with their support.

Of course, there are still many practical open questions- What happens with privately owned cabs? Would drivers be willing to shift their schedule? Will people tolerate fluctuating surcharges during parts of the day?

These ideas are just the start of the conversation and its going to take a combination of solutions to move the needle. If you can convince the TLC to release the data to the public, then we can really start to see the innovative spirit of New Yorkers as they take on this issue in new ways. And perhaps, with the help of Open Data, the rush hour taxi problem will soon just be part of our checkered past.



——-

-Raw data available here

-Example queries can be found here

-Graphs/Analysis done in Excel

-Thanks to Gunnar Larson, whom I met while speaking at TEDxBroadway, for posing this question to me. He has been discussing this with Gates Scholars.