Rupert Murdoch's media group News Corporation bowed to pressure from the public and parliament on Wednesday and withdrew its bid to take full control of pay-TV company BSkyB.

All three main political parties were poised to call on News Corp to abandon its offer in a vote in the House of Commons later on Wednesday.

The move leaves News Corp's key strategy for UK corporate growth in tatters. The proposed £8bn deal has been in train for more than a year, with the first offer tabled in June 2010.

It is the one of the biggest setbacks the 80-year-old media mogul has ever suffered and follows 10 days of revelations about the true scale of phone hacking at the News of the World, the paper Murdoch shut down last week.

The decision to abandon the deal is also a major blow to James Murdoch, who is third in command at the company and has responsibility for News Corp's UK businesses, including its Sky stake and News International.

It is likely to lead to criticism from investors over the way the company has handled the phone-hacking affair. James Murdoch initially took charge of the scandal but his father has twice flown in to the UK to oversee matters, most recently at the weekend.

News Corp's deputy chairman and chief operating officer, Chase Carey, said it had become clear that the Sky takeover "is too difficult to progress in this climate".

Carey, who is also News Corp's president, said: "We believed that the proposed acquisition of BSkyB by News Corporation would benefit both companies but it has become clear that it is too difficult to progress in this climate.

"News Corporation remains a committed long-term shareholder in BSkyB. We are proud of the success it has achieved and our contribution to it."

News Corp will have to pay BSkyB a break fee of around £38.5m after walking away from the deal. But Wall Street welcomed the withdrawal of the bid, sending News Corporation's A shares up by 4.3% to $16 in the first few hours of trading in New York.

After falling sharply following the announcement that the deal was off, BSkyB's share price then rallied strongly. The shares closed nearly 2% higher at 705.5p – above the 700p level at which News Corp originally tabled its bid. City traders said there was a sense of relief that the situation had been resolved.

BSkyB shares had been changing hands for 850p at the start of last week. Around £2.5bn has been wiped from the value of BSkyB since the Guardian revealed on Monday 4 July that News of the World journalists had hacked into a mobile phone belonging to murdered teenager Milly Dowler.

The decision to walk away from the deal was taken earlier on Wednesday before prime minister's questions, which was followed by an announcement by David Cameron about the details of two separate inquiries, one into phone hacking and the other into media standards.

BSkyB issued a statement noting the News Corp announcement and saying it believed it had "a compelling investment case and significant growth opportunities, as demonstrated by its excellent operational and financial performance and strong balance sheet which provides both strategic and financial flexibility".

"We remain very confident in the broadly based growth opportunity for BSkyB as we continue to add new customers, sell more products, develop our leading position in content and innovation, and expand the contribution from our other businesses," said Jeremy Darroch, the BSkyB chief executive. "I would like to commend all our employees for their unrelenting focus throughout the offer period and thank them for their continuing support."

Nicholas Ferguson, Sky deputy chairman and the senior independent non-executive director, added: "Since the start of the offer period, BSkyB's management team has remained fully focused on its strategic and operational priorities, as evidenced in the strong results reported for the first nine months of the financial year. With good momentum and a range of options for continued growth, BSkyB is well positioned to increase earnings and cash flow and deliver higher returns for shareholders."

Carey was at News International's Wapping offices on the fringes of the City of London briefly, where the decision is believed to have been finalised.

David Cameron, the prime minister, said News Corp should focus on sorting out the wrongdoing at News International.

"That must be the priority, not takeovers, so [withdrawing the BSkyB bid is] the right decision, but also the right decision for the country too. We've now got to get on with the work of the police investigation and the public inquiry that I have set up today," he added.

Nick Clegg, the deputy prime minister, said withdrawing the bid was the "decent and sensible" thing do to.

The Liberal Democrat leader briefly threatened to cause a coalition split when he declared Murdoch should abandon the Sky offer earlier this week, before Cameron decided he would also back a Labour motion to call for it to be dropped.

The shadow culture secretary, Ivan Lewis, said: "It's a victory for the public of this country, it's a victory for parliament and it's a victory for the tremendous leadership that Ed Miliband has shown."

• To contact the MediaGuardian news desk email editor@mediatheguardian.com or phone 020 3353 3857. For all other inquiries please call the main Guardian switchboard on 020 3353 2000. If you are writing a comment for publication, please mark clearly "for publication".

• To get the latest media news to your desktop or mobile, follow MediaGuardian on Twitter and Facebook