Rod Smith's job is to make sure IBM is focused on the nascent technologies that will change the world a decade from now. So perhaps it's no surprise that he has a special place in his heart for the big data projects IBM is running in partnership with police departments across the nation, crunching massive amounts of public information to try to predict where and when crimes will occur. The project, known as CRUSH — Criminal Reduction Utilizing Statistical History — has proven very effective in pilot programs in several American cities, including Memphis, Tennessee, where it been credited with reducing serious crimes by 30 percent and violent crimes by 15 percent. "We are giving these cops an inexpensive way to quickly test very big, complex hunches," said Smith. "So far, the results have been amazing, and this is just the tip of the iceberg."

Can you explain to me how this sort of predictive policing works?

Basically we're saying, how can you look at patterns of activity and extract data which is predictive for the near future. So detectives understand, if there is a series of burglaries, the robbers usually don't live more than a few miles from the house being targeted. They had a hunch that, just like a fingerprint, the thieves have a distinct pattern which repeats itself. We brought all that together with public data to determine when it might be a good idea to increase foot patrols in a certain area.

What are some of the key pieces of data you are looking at?

You look at the weather. Criminals are more likely to work when it's overcast. You look at traffic, they are going to want an easy escape route, not a crowded road. Some of the beat police asked us to look at parking, and hey, it turns out that criminals pay a lot of attention to this stuff, and don't go after a house when parking regulations are in effect, and traffic cops are likely to be in the area. You start to bake all these things together, and you can say, hmmm, this coming Wednesday, from 2:30-5, seems like a high risk time in this neighborhood, let's up our man power.

Where is this data coming from?

Well a lot of this stuff comes from public sources: traffic, weather, etc. But Twitter is showing that amidst these volumes of data, there are tiny nuggets of great value. It’s a public source of sentiment data way bigger and better than anything we have had before. If Facebook could open up API’s so we could understand people’s relationships better, it would be valuable, but its doubtful they will be doing that anytime soon.

You mentioned Twitter a lot during our conversation today. What makes that service such a powerful data source?

"Tweets are also much faster than the traditional alert systems."

Well it's intrinsically a public service, versus Facebook which is largely private. Tweets are also much faster than the traditional alert systems. When that recent earthquake hit Virginia last year, I think there were something like 4,000 tweets in the first eight seconds. It took the scientists monitoring this over twenty minutes before they released their first official report that it was an earthquake.

Do you think people are worried this might turn into some sort of Orwellian 'Minority Report' where cops are reacting to a crime that has not been committed?

We have a video commercial for this program that I just love. The bad guy shows up to rob a convenience store, and the cop is waiting there in the parking lot. So the criminal turns around and leaves. We're talking about preventing crimes. But hey, I think the best way to help the public understand the benefit, it to make the data open and transparent. You’ve got to make it transparent so people don’t think, that’s creepy, how did they do that? It can be an alert that the police share with citizens, almost like the weather, saying your area is currently at a higher risk of crime than normal, and here is the data that we're basing that on.

But if you make that data public, won't criminals use it against you?

Criminals are going to be as cagey as the public. They will change things very quickly. You have to be fast to find the new patterns. The processing of this information, when it gets to be that open, we will have to act faster then the bad guys.

Read more 5 Minutes on The Verge.