Senior MPs from both wings of the Labour party expressed fears last night that they could lose a snap general election to a reinvigorated Tory party led by Boris Johnson because of their own party’s internal feuding, indecision over Brexit, and failure to end the bitter controversy over antisemitism.

The Labour frontbencher Clive Lewis, who is on the left of the party, called on the leadership to “get a grip” before it was too late.

In a separate development, a further sign of Labour turmoil came last night when shadow education secretary Angela Rayner, a Jeremy Corbyn loyalist, broke cover to join fellow shadow cabinet member Jonathan Ashworth in voicing concern about Labour’s handling of antisemitism. The shadow cabinet was “totally exasperated by the failure of our movement to be able to deal with this issue”, she said.

In a striking intervention at a Fabian Society conference, Rayner said she was “absolutely embarrassed by what’s happened over the last few days and I’ve made my representations very clear. If that means setting up an independent system, then so be it... let’s just get on and do it.”

The comment came after a chaotic week for Labour in which shadow cabinet tensions over the party’s indecision on Brexit boiled over, and the Labour MP Chris Williamson was suspended again over his comments about antisemitism – just two days after being readmitted to the party.

Another shadow cabinet minister told the Observer that Labour had shed more than 100,000 members since its peak in 2016, and was now in financial difficulty. “The membership has dropped to around 400,000 [from more than half a million] and we are losing money,” he said. “We are not in a fit state to fight an election which I think we are heading towards.”

Facebook Twitter Pinterest Clive Lewis MP, right, on the pro-EU Put It to the People march in London. Photograph: Andy Hall/The Observer

Lewis said that with the Conservative party looking likely to elect a populist rightwinger, Boris Johnson, to replace Theresa May, the dangers and challenges were clear.

“Now more than ever our democracy needs Her Majesty’s Opposition to do what it says on the tin and oppose and act as a counterbalance to what a significant portion of the electorate will consider as a likely dangerous lurch to the hard right,” said Lewis.

“If Labour cannot perform this crucial function, then the danger for us as a party is that an increasing portion of the electorate will ask themselves: ‘If Labour won’t or can’t step up, then who will?’ The answer to this question may not be one we like very much. As such it really is time to get a grip and quick,” he added.

Yesterday, Corbyn emphatically denied that he is suffering from ill health, after the Times reported that senior civil servants were concerned he might have to stand down because he was not up to the job “physically or mentally”.

Responding to the paper’s front page article, Corbyn demanded an investigation into who in the civil service was spreading such claims. “The idea that civil servants should be briefing a newspaper against an elected politician, against a prospective government, is something that should be very concerning to all of us,” he said.

Facebook Twitter Pinterest MP Chris Williamson addresses a Labour rally. Photograph: Stefan Rousseau/PA

“The civil service has to be independent; has to be non-political and has to be non-judgmental of the politicians they have a duty to serve.”

Neil Coyle, a member of Labour’s parliamentary committee, which meets Corbyn each week, said that the leader and his team had compounded disquiet among Labour MPs by insisting last week that they had to make clear within the next fortnight whether they wanted to stand at the next election. This raised fears that some critics of Corbyn could face deselection in so-called “trigger ballots” in their local parties.

“When the party should be focused on strategy to win an election, staff will be dragged into these elections,” Coyle said. He added that Corbyn should now be thinking about his legacy: “It could be one of building Labour membership and a bold policy agenda or it could one of continuing fudge on Europe, with people leaving the party because of his failure to deliver.”

An Opinium poll for the Observer of 1,000 people who voted Conservative at the 2017 general election today shows 52% back Johnson to be the next Tory leader and prime minister, compared with 30% who support Jeremy Hunt.

Wes Streeting, Labour MP for Ilford North, said: “This has been another miserable week for the Labour party. More humiliation over antisemitism and still no clarity on our position on Brexit. It’s not just Corbyn’s critics who have had enough; it is clear from the shadow cabinet that his closest allies are at the end of their tether too. This isn’t sustainable.”

It has emerged that Williamson faces being deselected as an MP by local Labour activists should the party allow him back in the fold.

Plans are already in place to remove him as Labour’s candidate for Derby North before the next election. Williamson was suspended again on Friday just two days after being readmitted into the party. It followed an outcry from MPs, peers and staff.