A pro-EU Labour peer and former cabinet minister has said the party is “finished” if it contests another election promising to back Brexit, as a new poll suggested the party’s support depends heavily on remain voters who could switch their allegiance to the Lib Dems.

Prospect of a new UK party grows as ground shifts at Westminster | Heather Stewart Read more

Andrew Adonis, a former transport minister and a vocal critic on Britain’s departure from the EU, said the party could not be seen as an “accomplice to Brexit” should a snap election take place before March 2019.

A YouGov poll of more than 4,900 people, released to the Guardian on Sunday, put the Conservatives ahead of Labour by four points in a snap election should the latter adopt an anti-Brexit position, and ahead by nine points if Labour were to pledge to follow through with leaving the EU.

The Lib Dems would gain 10 points from Labour backing Brexit, lifting them to two points behind Labour.

“If Labour becomes an accomplice to Brexit, it is finished,” Lord Adonis said. “If people want Brexit, they will vote for the party that really believes in it, not the one that is being led to support it against its most profound convictions and misgivings.”

The poll was conducted for the anti-Brexit group Best for Britain and the anti-racism group Hope not Hate. The chief executive of Best for Britain, Eloise Todd, said: “These figures show that Labour would make real gains in the country from a clear anti-Brexit position. With Labour voters and national public opinion shifting against Brexit, it’s time for the opposition to give people what they want: a clear choice on Brexit between leaving and staying and reforming the EU, not a fudged deal that works for Westminster elites and no one else.”

Labour has said it will vote against any final Brexit deal unless it meets the shadow Brexit secretary Keir Starmer’s six tests on protection of jobs and rights, which in practice is highly unlikely. Downing Street has calculated that it will need the support of some Labour MPs if the deal has a chance of being passed in the Commons in November, and the Cabinet Office has given briefings to opposition MPs.

Jeremy Corbyn has said his party would push for a general election rather than a referendum on the deal should Theresa May fail to get her proposed agreement with the EU through parliament.

Party strategists have calculated that Labour retains the vast majority of remain supporters’ votes and that its electoral prospects are reliant on winning back or retaining the support of working-class leave voters.

Any move to soften the party’s Brexit position is likely to be fiercely opposed by some Labour backbenchers.

MPs from leave constituencies, including Caroline Flint and Gareth Snell, spoke out against party colleagues who had pushed for the party to support an amendment to keep the UK in a Norway-style single market deal during the EU withdrawal bill’s passage through the Commons this summer.

That vote saw Labour MPs split three ways, rebelling on both sides against a whip to abstain, exposing the party’s divisions over Europe.

Senior Labour figures are considering how to head off a concerted attempt by remain-supporting members to stage a vote at its annual conference calling for a second referendum. Around 130 constituency Labour parties are understood to be willing to back a motion in favour of a second vote, drafted by the pro-Corbyn campaign group Labour for a People’s Vote.