Liz Alter

Charles Duhigg is a staff writer at The New York Times and author of the coming book “The Power of Habit.” His cover article this week on how companies study your habits to know what you want before you do is adapted from the book. He also answered readers’ questions in a Facebook chat.



How did you first become interested in the story and get in touch with Andrew Pole, the Target statistician?

I knew I wanted to do a chapter in the book on how corporations study your habits, so I was calling around and someone mentioned to me that Andrew Pole had given a speech at a conference called Predictive Analytics World in which he said that Target was studying people’s habits in part to figure out whether women are pregnant. The speech was there online, and he did say that in a way that didn’t make it clear that it was a big deal.

When did Target tell him to stop cooperating with you?

I had long conversations with Andrew over a number of weeks. And then when I went to Target with questions they said there was no one available to meet with me. I sent them a list of questions about Andrew’s presentation, and the next time I talked to Andrew he said he’d been told not to talk to me anymore.

And that was the end of it? Was he apologetic?

I actually can’t go into the details of what happened because we are getting into the area of confidentiality. But Target intervened, and Andrew Pole stopped returning my phone calls.

Do you have any sense of whether Target is representative of other companies? Are they the most egregious, or the most skilled at this sort of thing?

Every single company does some version of this. If you are a large company and you are not involved in analytics, you are not a large company for very much longer. Target is an interesting example because they are among the smartest companies at doing this stuff. Everyone can collect data. The question is, Can you come up with clever ways to analyze the data? Can you come up with clever questions? It might not occur to every company that there is a way to parse the data to find pregnant women.

That most clever of techniques obviously pushes people’s buttons. Felix Salmon’s response to your article was that Target’s defensiveness was fascinating precisely because we haven’t quite figured out where the creepy/not creepy line is.

And that line is moving. People have become more comfortable with giving their data away. If you had told people 10 years ago that companies would be observing everything you do online and they would make assumptions about your behavior as a result, they would freak out. And now when you tell them, they say, “I figured that was happening.” Felix brought up the point that sometimes we want companies to be smart about the coupons they send us because we want them to send us coupons we need. If I’m in the market for a refrigerator, I don’t want coupons for earmuffs.

Pregnancy seems pretty clearly off-limits. Are other things also still off-limits?

Pole told me that it is more art than science. For instance, if customers knew that credit-card companies were trying to figure out whether they were headed toward divorce, that would freak people out. And in fact credit-card companies keep an eye on whether you are making purchases of that kind, because divorce is expensive and they are paranoid that you might stop paying your credit-card bill. For example, if you use your card to pay for a marriage counselor, they might decrease your credit line. Or if you do that and then make a change in your payment timing, they might call you to remind you of your payment schedule. When companies can see inside our bedrooms or inside our wombs, it seems too personal. But they can see inside our shopping carts, and people are reconciled with that.

There’s always the risk of blowback. Several women who commented on your story said they continued to receive pregnancy-related mailers for months after a miscarriage.

That’s heartbreaking. When you are doing this kind of analytics, which is called “big data,” you are looking at hundreds of thousands to millions of people, and you are converging against the mean. I can’t tell you what one shopper is going to do, but I can tell you with 90 percent accuracy what one shopper is going to do if he or she looks exactly like one million other shoppers. You expect that there is some spillage there, and as a result that you will give the wrong message to a certain number of people. Women who have had miscarriages are a tragic version of that.

Do you think that people are right to be freaked out?

I’m a bit of an agnostic on that question. Should people be freaked out by capitalism? If we had the luxury of choosing between a lot of different systems, that would be a legitimate question. But capitalism is the system that we’ve got. The truth of the matter is that we don’t have the option to decide whether a company like Target can use data. Everyone is already collecting data and using it all the time. And our ability to amass and use data has revolutionized everything from public health to the way we conduct political campaigns. President Obama just hired a habits expert who used to consult for grocery stores on this stuff to diagnose and learn how to trigger individuals’ voting habits. This science is just science, just a tool. It can be used for good or in ways that make us uncomfortable.