It’s a bright Thursday morning in Brexit-voting Medway, and Jeremy Corbyn is launching Labour’s campaign for the European elections by explaining that it’s not a remain party – or a leave one.

“We could allow ourselves to be defined only as ‘remainers’ or ‘leavers’ – labels that meant nothing to us only a few years ago. But where would that take us? Who wants to live in a country stuck in this endless loop?” he asks his audience.

A handful of activists have been arranged into a line along the Labour leader’s path to the podium. They clap as he passes, making an upbeat backdrop for the broadcast shots. It’s not a bad turnout on a weekday morning, for an election no one, least of all Corbyn, wanted to fight.

The mood is lively and the applause warm, if not quite as ecstatic as in the heady days of 2017, and Corbyn seems relaxed.

Rochester and Strood was the constituency of Mark Reckless, the Conservative MP who defected to Ukip and won a byelection in 2014. And while the Tories won the seat back in 2015, 63% of voters here chose Brexit in the referendum.

Facebook Twitter Pinterest Jeremy Corbyn with local candidate Ruth Jones in Newport, Gwent. Photograph: Sean Smith/The Guardian

So the choice of this area to launch Labour’s campaign is significant: Corbyn wants Labour’s message to resonate deep in Brexit Britain, as well as in the cities and towns some in his inner circle call Remoania.

“Labour will never be the party of the 52% or of the 48%,” he says. “We are the part of the great majority who reject the politics of smear and scapegoating.”

He bashes bankers, corporations and of course the Tories, and in response to questions about Labour’s position on a referendum, says he hopes that if it did happen, it could be a “healing process”.

These past three years certainly haven’t been a healing process for Corbyn’s fractured party – or for the leftwing movement that propelled him to the Labour leadership.

If the polls are correct, Labour will be punished harshly on Thursday for his determination to play to both sides of the bitter Brexit divide in British politics. They could even face the humiliation of being overhauled by Vince Cable’s reinvigorated Lib Dems.

The question exercising the party’s activists, MPs and candidates is how lasting the damage is likely to be – and what it means for Corbyn and his radical project.

Facebook Twitter Pinterest Jeremy Corbyn at the Gurdwara Sri Guru Singh Sabha Sikh temple. Photograph: Sean Smith/The Guardian

Matt Zarb-Cousin, a former spokesman for Corbyn, says: “I think it’s an incredibly difficult position for them, and they’re always going to get squeezed, because they’re having to hold together an election-winning coalition of leavers and remainers.

“It’s easy for the Brexit party or the Lib Dems, because the Lib Dems don’t have any aspiration to form the next government, so they can have a single-issue campaign.”

Pollsters disagree about whether there is any sound political rationale for Corbyn’s careful balancing act.

Peter Kellner, formerly of YouGov, argues that remain voters who backed Labour in 2017 are abandoning it, and the party’s strategy is “specifically haemorrhaging remain votes without enhancing its appeal to leave voters”.

But Ian Warren, of the consultancy Election Data, who worked inside Labour during the 2015 general election under Ed Miliband, says that doesn’t mean it would be right for Labour to jump to an out-and-out remain strategy.

He argues that Brexit has exacerbated what was already a formidable long-term challenge for the party, of stitching together a coalition of voters who could ultimately carry Corbyn into Downing Street.

“When Labour MPs now are pushing back against a second referendum, what they are pushing back against is the idea that the voters who the party lost in the early 2000s can just be forgotten about,” Warren says. “They are the people that the party was built to represent: traditional working-class voters. So if it isn’t representing those groups any more, or it’s choosing a path that is against the wishes of those people, then what is it for? It’s an existential question.”

“What I’ve always warned the party would happen has started to happen, which is it’s lost its traditional base, going after newer, liberal metropolitan voters – and now it’s losing them,” he says. “Brexit accelerates that, because it provides people a very real, time-limited test for their loyalties.”

That certainly rings true for the Leeds North West MP Alex Sobel. On a sunny Monday afternoon in leafy Headingley, he is out knocking on doors with a keen team of activists, together with the second candidate on Labour’s list for this region, Eloise Todd.

Facebook Twitter Pinterest Alex Sobel campaigning in Headingley, Leeds. Photograph: Gary Calton/The Observer

“The most common phrase we hear on the doorstep is: ‘I’d like to vote Labour, but …’,” says Sobel.

He has only been an MP since 2017, after unseating the Lib Dem Greg Mulholland, and he warns that it is Labour’s newer supporters – the ones who were inspired to join the party by Corbyn’s leadership – who are most frustrated by the party’s balancing act on Brexit.

“The thing that has surprised me most in this election is Labour party members, and strong supporters, people I know, who have either said to me or written to me and said, ‘I am or I was a very strong supporter of Jeremy Corbyn, but actually I’m going to vote another way in this election’ – usually Green.”

He adds: “The 2017 manifesto united the majority of Labour supporters and the Labour party behind that agenda – and we are frittering that away because of our line on Brexit.”

Sobel is one of a group of Labour MPs who have been criss-crossing the country under the banner of “Love Socialism, Hate Brexit”, rallying supporters to the kind of robust, positive, pro-remain message they would have liked to see Corbyn espouse.

Clive Lewis, the Norwich South MP who is one of the group’s founders, says: “I think the frustration of so many members, especially those who have supported the political project for the last three years, is that they look at the biggest ringleaders for leaving the EU, they look at the dark money, they look at all those far-right organisations across Europe who are cheering Brexit on, and they think: ‘Why isn’t the Labour party standing up to these fascists and authoritarians? If they think Brexit is good, why aren’t we against it?’”

Facebook Twitter Pinterest Clive Lewis. Photograph: Graeme Robertson/The Guardian

Friends say Corbyn himself has softened somewhat towards the notion of a referendum in recent days and weeks – in part because he can feel the pressure building up in his own Islington North constituency Labour party (CLP) for a cleaner, pro-remain position.

“What people need to understand about Jeremy, he’s got a Spanish-speaking wife [Laura Alvarez is Mexican], he’s got a son in his early 30s, and his CLP love him, but they are overwhelmingly remain,” says one colleague who knows him well. “But he also spends more time on the road than probably any other senior British politician – so he’s got his adoring fans in Islington, who tell him we have got to have a second referendum, but he also then goes to the parts where people are really pissed off that Brexit is going to be stolen from them. And what he desperately wants to do is to hold it all together.”

In an interview with the BBC’s Andrew Marr, recorded in Liverpool and broadcast on Sunday, he appeared warmer towards the idea of a referendum on any deal that passes through parliament, calling it a “reasonable” idea.

This video has been removed. This could be because it launched early, our rights have expired, there was a legal issue, or for another reason. Jeremy Corbyn says he 'won't support new attempt at Brexit deal' - video

His words were seized on by anxious Labour remainers. Todd, the MEP candidate for Yorkshire and the Humber, chatting the next day over coffee before hitting the doorsteps in Headingley, says: “I think Jeremy was clear yesterday. It was the first thing out of his mouth, and it felt like it was something he wanted to say.”

She is on unpaid leave from her job running the anti-Brexit campaign group Best for Britain, and says some of her remainer friends have pressed her on why she is running for Labour, and not the “bollocks to Brexit” Lib Dems.

“Personally, I’m very, very committed to Labour’s approach of bringing the two sides together,” she explains. “The worst thing you could have is middle-class remainer parties and working-class leave parties. We are the only party that can bring people together.”

But she adds: “Where you can’t fudge is you can’t have two destinations. You have got to say you’re planting your flag and you’re going somewhere.”

Facebook Twitter Pinterest The Labour candidate Eloise Todd in Headingley. Photograph: Gary Calton/The Observer

Labour sources were quick to insist on Sunday that Corbyn’s line did not represent a shift in position – prompting irritation among members of the shadow cabinet, one of whom told the Guardian: “I saw the Marr interview. They weren’t throwaway remarks. And in the context of Brexit, JC rarely says anything he hasn’t thought about and doesn’t mean.”

Another joked that Corbyn was at risk of being “chained to a radiator” by Eurosceptic advisers. But several figures at Labour’s top table remain deeply sceptical about the idea of a referendum.

The shadow Cabinet Office minister, Jon Trickett, who is part of Corbyn’s inner shadow cabinet, with Diane Abbott and John McDonnell, says: “Many voters have told me that their faith in the whole political system is at a low ebb. Clearly there is a risk in the idea of a second referendum that the whole political system will lose the people’s consent. If that were to happen, the country would be faced with a major crisis.”

On Sunday, Corbyn was back on home turf, with a kind of whistlestop tour of London’s cultures, from a Tamil event in the morning, via a Sikh gurdwara where he helped serve lunch, to the vast London Central mosque, where he addressed Muslim leaders.

It was then packaged up into a cheery video to be shared on social media, saying: “Cultures coming together are a wonderful thing … it’s a joy, not a problem.”

It felt almost like a conscious effort to recapture the hopeful days of 2017. Part of Corbyn’s appeal at that general election was Labour’s manifesto, with its bold pledges to abolish tuition fees, renationalise the railways and boost the minimum wage.

Much of it though came from Corbyn himself – the unshowy, man-on-the-allotment ordinariness, alloyed with earnest enthusiasm for the issues he really cares about: poverty, foreign policy, immigrants’ rights.

Yet the action on Brexit has of necessity taken place in Westminster, not out on the road (or more accurately, the rails), where Corbyn is visibly happier. And it is harder to appear as the scourge of the establishment when your lieutenants are eyeballing Theresa May’s negotiators across the table in Whitehall.

Facebook Twitter Pinterest Labour election leaflets. Photograph: Gary Calton/The Observer

In Leeds, Sobel is concerned that the newly enthused voters who helped send him to Westminster two years ago will not support Labour again.

“They feel like they’re sending Jeremy a message by not voting for us. And these are people that voted for him twice in the leadership elections – and I don’t really know what to say to them,” he says, despairingly. “I really want him to speak to them, because they’re in the Labour party because of him, not because of me.”