The Scottish parliament is to be handed direct control over billions of pounds of income tax and welfare benefits after an £11bn cross-party deal which will lead to the biggest shakeup to Britain’s taxation system in the modern era.

The historic move, which had been resisted by the Labour leadership in London, means powers to set income tax rates and bands are to be wholly devolved to Holyrood as the pro-UK parties move to outflank the Scottish National party, which has surged in popularity since the referendum on Scottish independence.

High-level talks to agree the full package of new powers continued late into Wednesday as negotiators from all the main parties at Holyrood thrashed out the details of a deal due to be revealed in Edinburgh on Thursday morning.

The far-reaching reforms are due to include £3bn worth of welfare powers, including the housing elements of universal credit, attendance allowance and carers allowance, the work programme and winter fuel payments, as well as air passenger duty. Along with the income tax powers, the total package is estimated by Treasury officials on the commission to be worth about £11bn.

But the deal led to an immediate backlash in England with a cross-party call from all the senior figures in English local government for a comparable devolution process in England. In a joint letter, officials including London’s mayor Boris Johnson and six Labour leaders of English city councils called on the UK government to set up a similar commission to agree a “comparable package of measures for local government in England”.

The Smith Commission, which was set up after pledges by the UK’s three main party leaders in the last few days of the Scottish referendum campaign, is preparing to publish its heads of agreement.

Lord Smith of Kelvin, the head of the commission, has been working with Scotland’s five main parties – Labour, the SNP, the Tories, the Liberal Democrats and the Greens – to reach broad agreement by St Andrew’s Day on 30 November in line with the timetable laid down by former prime minister Gordon Brown.

In the most significant move, Labour has abandoned its opposition to the full devolution of income tax to bring it into line with the four other parties in a bid to halt its plummeting poll ratings in Scotland in the face of a surge for the SNP.

Brown had described the proposal as a Tory trap while Alistair Darling, the leader of the pro-UK Better Together campaign in the referendum, warned in the Financial Times this week that it would end in “floods of tears” because it would undermine the principle that risks are shared across the UK.

But Ed Miliband and Ed Balls, who are expected to offer broad support for the commission while saying the details will need to be studied with care, have decided to bite the bullet to avoid a further slide in Labour support in Scotland.

It is understood that Balls won two important concessions that will give comfort to the Labour leadership in London. The personal tax allowance – the amount of income people can earn before they pay tax – will remain a UK matter that will continue to be decided by the chancellor of exchequer and voted on by MPs at Westminster.

This will make it impossible for a Scottish finance secretary to do what chancellors have done in the past – announce an income tax cut and pay for it by changing the personal allowance.

In the second key concession, it is understood that the commission will say that there is still a UK income tax system. It is also expected to say there should not be two classes of MPs at Westminster. This is designed to guard against the Tories undermining a Labour chancellor, who may depend on the votes of Labour MPs from Scotland, by saying that he or she should hold a budget for the whole of the UK and a separate budget for England, Wales and Northern Ireland.

The deal is expected to allow Scottish ministers to control the housing benefit elements of the new universal credit system, potentially worth about £1.7bn. The powers, agreed during secret talks overseen by Smith and officials from the Treasury, are expected to be fast-tracked through Westminster next year.

Labour has been forced to give ground on devolving air passenger duty and on allowing Holyrood full control over Scottish parliament and council elections – including allowing 16- and 17-year-olds full voting rights.

But as Labour, the Tories and the Lib Dems edged closer to a deal with the SNP and Scottish Greens, a controversial plan to devolve abortion law to Scotland, ending a UK-wide approach to cut-off dates on terminations, caused divisions.

Backed by Labour negotiators, leading women’s groups including Abortion Rights and Engender Scotland submitted formal protests, leading to the plan being dropped in the final stages. Instead, the commission is expected to state that it is minded to devolve abortion law, but to agree that further work is needed.

The Scottish Greens, the SNP and Smith support devolving abortion legislation and are understood to have argued that Holyrood has taken a liberal, pro-equality stance on all other policies on women’s health and equalities.

But critics claim that no evidence has been found to justify the move and that there was no public debate before it was tabled by the Scottish Greens. The groups said it risked giving anti-abortion campaigners the power to restrict abortions and creating a two-tier system in Britain.

Maria Fyfe, a leading pro-choice campaigner and former Labour MP for Glasgow Maryhill, said there had been an astonishing lack of consultation.

“There is a huge danger that this will increase attacks on abortion rights if the legislation is devolved,” she said. “Take the likes of Brian Souter, a big funder of the SNP, who strongly opposes abortion rights and funded an anti-gay rights campaign. How can we not be suspicious?”

Carolyn Leckie, a former midwife and senior figure in Women for Independence, said devolving abortion would allow the pro-choice movement to make Scottish abortion law more progressive and more flexible than the rest of the UK. The Smith Commission’s decision to allow Scotland to set its own air passenger duty to stimulate business travel and tourism will infuriate airports in northern England.

Newcastle airport protested about the SNP’s proposal to cut and then abolish the duty during the referendum campaign, fearing it would cut passenger numbers. The measure was backed by Ryanair and British Airways.

Labour was forced to drop proposals to take full control over housing benefit because it is an integral part of the universal credit single welfare payment. Instead, Holyrood is expected to be allowed to vary housing costs and the frequency of payments and be able to direct payments to either the claimant or housing provider.

Labour sources said these concessions proved the party was listening to Scottish voters, who came close to backing independence. “We needed to show good faith and show we understand that people voted for change,” said one.

The SNP is expected to accept all the Smith proposals, but will insist that Scotland needs far greater autonomy, particularly over taxation and welfare.

The Labour leadership will move to avoid an English backlash against favourable treatment for Scotland when Balls says that his devolution to England will see £30bn in budgets given to English city regions, as well as new powers to keep more of the business rate.

In a speech in Birmingham, Balls plans to say: “The next Labour government will radically devolve power and £30bn of funding over a parliament to city and county regions in every part of England. Devolution not just to cities, but across all our towns and county regions, too.”

“We will give groups of local authorities substantial new powers over back-to-work schemes, to drive house building, and to integrate, invest in and plan transport infrastructure.

“And a Labour Treasury will also allow city and county regions that come together in combined authorities to keep all the additional business rates revenue generated by growth.”