After a nationwide crackdown on men who buy sex in the 8th National Day of Johns Arrests earlier this week, comedian Jim Norton wrote an essay asserting his right to pay prostitutes for sex, called “In Defense of Johns.”

Don’t get me wrong, Norton is a funny guy. And I’m all for comedians pushing our social limits in stand-up, because that’s what comedy is all about. But why can’t a famous comic like him find someone who wants to have sex with him for his good looks and sparkling personality?

Norton’s essay wasn’t a joke — it was an actual argument defending the right to pay for sex. “But really, perhaps the most shameful thing I can admit is this: I’m not really ashamed,” he wrote. “And neither should any of these other (unmarried) johns who have been arrested.”

Actually, Jim, you should be ashamed to pay for sex. And so should all the other men who purchase women and girls, many of whom have been trafficked, enslaved and repeatedly raped. No amount of rationalization can get around the basic principle of market economics: if people like you didn’t buy girls, they wouldn’t be sold, and if they couldn’t be sold, they wouldn’t be trafficked and abused.

(Of course, there are also women who buy sex, and plenty of men and boys who are trafficking victims, but let’s focus on the male-client/female-sex-worker argument that Norton is going with.)

There was one part of Norton’s essay that I did find funny. It was the part where he said all the girls he buys are oh-so lucky to be with him. “I suppose you could say I am the consummate john,” he wrote. “I’m loyal, I’m dedicated and I will always come back.” He’s different from all those other nasty, mean clients, because he’s a really nice guy! “I never pick them up to be abusive,” he said. “I always feel extraordinarily loving and close to them.” Hahahahahaha, Jim Norton. Good one!

Did you ever consider, Jim, whether these girls felt “extraordinarily loving and close” to you? I’m guessing their feelings were a bit more complicated. They might have slept with you only because they would get beaten if they didn’t make a certain amount of money that night. And if you thought they enjoyed it, they were probably faking, because that’s exactly what you pay them to do. Sure, some woman do choose this line of work, and sex-workers unions argue that prostitution can be a freely made choice, but that’s not the case for the vast majority: U.S. State Department estimates that 80% of the 600,000 to 800,000 people trafficked across international borders every year are trafficked for sex.

And while we don’t know what the prostitutes thought of Norton, we do know what some sex workers say about their clients. One former prostitute named Kira put it this way: “You guys think we really liked having sex with you, but we would lie to you just to get your money … I hated you when I was out there,” she told men who had been busted for buying sex, according to PBS.

Men like Norton think that their entitlement to sex trumps a woman’s entitlement to dignity and safety. Many of the women they buy are among the most vulnerable human beings on the planet, no matter how wide they smile when a john rolls down his window or plunks down his credit card. According to a report cited by the U.S. State Department, 89% of people who work in prostitution worldwide want to escape. At least 65% of people who work in prostitution were sexually abused as children, and over 60% are raped on the job, according to a 2004 study by Melissa Farley, an activist and psychologist who studies the effect of prostitution on women. And according to Polaris, a Washington, D.C.–based antitrafficking group, over 40% of people trafficked for sex are under 18. Norton says he’s spent the “equivalent of a Harvard Law School education” on sex, which is precisely what keeps trafficking victims in the sex trade.

Norton claims that legalizing prostitution would help solve these problems, but what he really means is that it would be easier for him to buy sex without his pesky conscience getting in the way of his peskier penis. Because even though there are valid arguments for the legalization of prostitution, I’m finding it hard to believe that Norton really has the best interests of sex workers in mind.

Because despite the theories, there’s very little evidence that legalizing prostitution makes life better for sex workers. Even though prostitution is legal in Nevada, over 80% of the sex workers Farley interviewed told her they wanted to escape sex work. And five years after prostitution was legalized in Germany 2002, the Family Ministry found “no solid proof to date” that the legalization had reduced crime and abuse, and had “not brought about any measurable actual improvement in the social coverage of prostitutes,” according to the German newsmagazine Der Spiegel. Proponents say that legal prostitution can be regulated to ensure the safety of the sex workers, but German snack bars have more regulations than brothels do.

The Netherlands has also been held up as an example of what happens when prostitution is legalized, but the results are mixed. The mayor of Amsterdam said in 2003 that legalizing prostitution had failed to keep sex workers safe, since “it appeared impossible to create a safe and controllable zone for women that was not open to abuse by organized crime.”

Most arguments for legalization presume that tons of women would choose sex work if it were safe and legal, but that’s convenient wishful thinking for johns who want to let themselves off the hook. “In the real world, Julia Roberts’ character from Pretty Woman does not exist,” said Cook County Sheriff Tom Dart, who organizes the National Day of Johns Arrests and advocates for harsher punishment for sex buyers. “Every time a john purchases sex, he is catalyzing a violent and oppressive industry.”

“The autonomous prostitute we envisioned when the prostitution law was enacted in 2001, who negotiates on equal terms with her client and can support herself with her income, is the exception,” German politician Thekla Walker said at a political convention. Instead, the law allows sex workers “merely the freedom to allow themselves to be exploited,” according to Der Spiegel.

Some argue that making prostitution legal could make sex workers safer, because they could call the police if a client was getting violent. But criminalizing the johns would do the same thing: prostitutes would know they won’t face jail time for calling for help, and the violent jerk would be cuffed.

That’s why targeting the johns is the best way to keep vulnerable women safe. Since Sweden introduced a measure in 1999 to target clients instead of sex workers, the population of prostitutes has been reduced by two-thirds, from 2,500 in 1998 to just 1,000 in 2013. France recently did the same, imposing fines for men who pay for sex. And even New York City prosecutors are increasingly focused on targeting buyers and pimps instead of sex workers. Because women and children will be sold as long as there are men to buy them, and when the demand for paid sex outstrips the supply of willing prostitutes, traffickers are ready to step in.

Prostitutes have been shamed and marginalized for thousands of years, but men who buy sex are considered so normal that they’re given the most ordinary name of all: john, a name shared by no less than five U.S. Presidents. Imagine the name whore was as common as john, and you’ll see how ridiculous this is — think about “Whore Quincy Adams” as our sixth President. Let’s hope we see the day when the men who choose to buy sex are shamed as much as the women who are often forced sell it. They’re the ones that should be ashamed of themselves.

Write to Charlotte Alter at charlotte.alter@time.com.