'I don't get anything out of sex with prostitutes except for a bad feeling," says Ben. An apparently average, thirtysomething, middle-class man, Ben had taken an extended lunchbreak from his job in advertising to talk about his experiences of buying sex. Shy and slightly nervous, he told me, "I am hoping that talking about it might help me work out why I do it."

I, too, was hoping to understand his motives better. Ben was one of 700 men interviewed for a major international research project seeking to uncover the reality about men who buy sex. The project spanned six countries, and of the 103 customers we spoke to in London – where I was one of the researchers – most were surprisingly keen to discuss their experiences.

The men didn't fall into obvious stereotypes. They were aged between 18 and 70 years old; they were white, black, Asian, eastern European; most were employed and many were ­educated beyond school level. In the main they were presentable, polite, with average-to-good social skills. Many were husbands and boyfriends; just over half were either married or in a relationship with a woman.

Research published in 2005 found that the numbers of men who pay for sex had doubled in a decade. The ­authors attributed this rise to "a greater acceptability of commercial sexual contact", yet many of our ­interviewees told us that they felt ­intense guilt and shame about paying for sex. "I'm not satisfied in my mind" was how one described his feelings after paying for sex. Another told me that he felt "disappointed – what a waste of money", "lonely still" and "guilty about my relationship with my wife". In fact, many of the men were a mass of contradictions. Despite finding their experiences "unfulfilling, empty, terrible", they continued to visit prostitutes.

I interviewed 12 of the men, and found it a fascinating experience. One told me about his experience of childhood cruelty and neglect and linked this to his inability to form close ­relationships with anyone, particularly women. Alex admitted sex with ­prostitutes made him feel empty, but he had no idea how to get to know women "through the usual routes". When I asked him about his feelings ­towards the women he buys he said that on the one hand, he wants ­prostitutes to get to know and like him and, on the other, he is "not under ­delusions" that the encounters are anything like a real relationship.

"I want my ideal prostitute not to behave like one," he said, "to role-play to be a pretend girlfriend, a casual date, not business-like or mechanical. To a third person it looks like we're in love."

I felt compassion for Alex. No one had shown him how to form a bond with another human being and he was searching for something that commercial sex was never going to provide.

But another of the interviewees left me feeling concerned. Darren was young, good-looking and bright; I asked him how often he thought the women he paid enjoyed the sex. "I don't want them to get any pleasure," he told me. "I am paying for it and it is her job to give me pleasure. If she enjoys it I would feel cheated." I asked if he felt prostitutes were different to other women. "The fact that they're prepared to do that job where others won't, even when they're skint, means there's some capability inside them that permits them to do it and not be disgusted," he said. He seemed full of a festering, potentially explosive misogyny.

When asked what would end ­prostitution, one interviewee laughed and said, "Kill all the girls." Paul told me that it would take "all the men to be locked up". But most of them told the researchers that they would be ­easily deterred if the current laws were implemented. Fines, public ­exposure, employers being informed, being issued with an Asbo or the risk of a criminal record would stop most of the men from continuing to pay for sex. Discovering the women were ­trafficked, pimped or otherwise coerced would appear not to be so ­effective. Almost half said they ­believed that most women in prostitution are victims of pimps ("the pimp does the ­psychological raping of the woman," explained one). But they still continued to visit them.

An upcoming new law will make it illegal for men to pay for sex with a trafficked or pimped woman – and a punter's ignorance of a woman's ­circumstances will be no defence. Critics have suggested that this is ­unfair, that a man can't possibly know whether a woman is being exploited. Our interviews challenged this ­notion. The men knew, to some extent, about abuse and coercion in prostitution – they weren't operating under the ­convenient illusion that women enter the trade because they love sex. More than half admitted that they either knew or believed that a majority of women in prostitution were lured, tricked or trafficked.

More than one third said they thought the prostitutes they visited had been trafficked to London from another country, and a small number said they suspected that they had ­encountered a trafficking victim based on the woman's inability to speak the local language or on how young or vulnerable they appeared. "I could tell she was new to the country," said one man. "To be new in a country and be a prostitute – it can't be a choice . . . She looked troubled."

Another said that he had "seen women with bruises, cuts and eastern European accents in locations where lots of trafficked women and girls are". One man suspected that an African woman he had met was ­trafficked ­because "she was frightened and ­nervous. She told me she had been tricked. I had sex with her and she seemed fine with the sex. She asked me to help her, but I said there was little I could do. She might have been lying to me."

One of the most interesting findings was that many believed men would "need" to rape if they could not pay for sex on demand. One told me, "Sometimes you might rape someone: you can go to a prostitute instead." Another put it like this: "A desperate man who wants sex so bad, he needs sex to be relieved. He might rape." I concluded from this that it's not feminists such as Andrea Dworkin and myself who are responsible for the idea that all men are potential rapists – it's sometimes men themselves.

Half of the interviewees had bought sex outside of the UK, mostly in Amsterdam, and visiting an area where prostitution is legal or openly advertised had given them a renewed dedication to buying sex when they returned to the UK. Almost half said that they first paid for sex when they were below the age of 21. "Dad took me and my older brother," said David. "He paid. Maybe he wanted to make sure we weren't gay. We went to a brothel. Dad didn't do it, and I don't think he told my mum."

Another man paid for sex during a stag trip to Thailand with eight of his friends. He was disappointed. "It was a Russian girl, it wasn't the ­escort experience. She didn't want to talk, just lay on the bed and wanted to do the [sex] act only."

Many men seemed to want a real relationship with a woman and were disappointed when this didn't develop: "It's just a sex act, no emotion. Be prepared to accept this or don't go at all. It's not a wife or girlfriend." ­Others were clear that they paid for sex in order to be able to totally control the encounter, including Bob, who said, "Look, men pay for women because he can have whatever and whoever he wants. Lots of men go to prostitutes so they can do things to them that real women would not put up with."

Although some of the men said they thought the women they bought ­enjoyed the sex, many others admitted that they thought the women would be feeling "disgusted", "miserable", "dirty" and "scared". Ahmed said he thought the woman might feel "relief that I'm not going to kill her".

Only 6% of the men we spoke to had been arrested for soliciting ­prostitutes. "Deterrents would only work if ­enforced," said one. "Any negative would make you reconsider. The law's not enforced now, but if any negative thing happened as a consequence it would deter me." Perhaps the new law will make Albert think twice about paying for sex. He told me, "If I'd get in trouble for doing it, I wouldn't do it. In this country, the police are fine with men visiting prostitutes."

Read the research project's report here (pdf)

All names have been changed. Why do you think men pay for sex? Do you think more should be done to stop them? Email your views to women@theguardian.com or write to Women, The Guardian, Kings Place, London N1 9GU