Sir Philip Green faces a £3m legal bill after the high court formally allowed him to abandon his action to prevent the Daily Telegraph from publishing allegations of bullying and sexual harassment against the Top Shop tycoon.

The Telegraph estimated Green’s bill to be about £3m after the judge told him to pay the bulk of the newspaper’s costs.

On Friday night, the Telegraph published allegations that Green had subjected people working in his business empire to abuse that was at times racial, physical and sexual. It said some of the people involved had made complaints about Green but they had been covered up. In some cases, Green is alleged to have paid people large sums of money in return for their silence.

He was accused of making racist remarks to black employees, of groping female employees and of being physically aggressive and abusive towards both male and female members of staff. Green told the paper he denied any “unlawful racist or sexual behaviour”.

His lawyers said he is a “passionate businessman, who can at times be overexuberant and hot-headed”. They said he can be “perceived at times as aggressive with senior and trusted staff”. They denied this amounted to criminal behaviour.

Earlier, the Telegraph also published excerpts from six phone conversations with Green discussing the newspaper’s intent to publish a story based on the allegations. During the calls Green said that if his business were impacted by the revelations the ensuing legal action would see Chris Evans, the editor of the Telegraph, “need a new job and your paper might end up bankrupt as well”.

In the recordings Green describes Claire Newell, the Telegraph’s investigations editor, in disparaging terms including repeatedly calling her a “girl”.

“Your girl has run about all over the fucking place,” he says. “I can give you the list of a lot of the people she’s called on and she has found fuck all. Nothing. Zero. You’re telling me she’s your boss. Why are you phoning? Why hasn’t she got the balls to get on the telephone then?”

Green had been granted a temporary injunction blocking the Daily Telegraph from publishing allegations made by five employees, who had all received substantial payments and signed non-disclosure agreements after settling their claims.

Green said last week that he wanted to drop the case because it was pointless after the Labour peer Peter Hain had used parliamentary privilege to name him in the House of Lords. Justice Warby granted Green permission to discontinue the proceedings in a ruling on Friday.

Green and the board of his Arcadia Group said in a statement that they were pleased with the high court judgment.“The Telegraph has pursued a vendetta against Sir Philip Green and the employees and management of Arcadia Group for the past nine months, harassing many of its staff and their families at their homes, often at night and at weekends.”

The Telegraph had attempted to oppose Green’s application to discontinue the court case unless he agreed not to pursue the five individuals. In the judgment on Friday, the court allowed Green to drop the case but refused to offer any legal protection to the employees if their allegations were made public by the Telegraph.

He called on the Telegraph and its owners, the Barclay brothers, to do the “decent thing” and respect the non-disclosure agreements the former employees had signed and not publish the story.

Sign up to the daily Business Today email or follow Guardian Business on Twitter at @BusinessDesk

“If not they will expose their sources to potential further legal actions and significant losses,” the statement said. Green has “categorically and wholly denies” any suggestion he was guilty of unlawful behaviour.

“We are delighted the injunction has been lifted, but our campaign against the misuse of NDAs goes on,” said Evans. “In the wake of the Harvey Weinstein affair, we became aware that gagging orders called NDAs were being used to cover up allegations of sexual misconduct and racial abuse in the workplace. And that led to our investigation into Sir Philip Green and Arcadia. We maintain there is a clear public interest in telling people whether a prospective employer has been accused of abuse.”

Theresa May said in October that the government was committed to reforming the use of non-disclosure agreements. “Non-disclosure agreements cannot stop people from whistleblowing, but it is clear that some employers are using them unethically,” she said during prime minister’s questions.

She said the government was going to bring forward its consultation “to seek to improve the regulation around non-disclosure agreements and make it absolutely explicit to employees when a non-disclosure agreement does not apply and when it cannot be enforced”.

Evans called on May on Friday to deliver on those comments. “The prime minister has already indicated that she is uneasy with the way in which NDAs have been used,” he said. “We ask her now to do something about it.”