One of the wisest, if least obeyed, maxims of political debate is: don’t argue about facts, check them out instead. In the case of Labour’s strategy towards next week’s European parliament elections, one central fact is now beyond dispute. The recent slump in the party’s support has been caused by the desertion of voters who want the UK to stay in the European Union: not partially caused, or possibly caused, but totally.

This is clear from a detailed analysis of recent YouGov surveys by its political team. In common with other pollsters, it has picked up a sharp reduction in Labour’s support, from 21% in late April to 16% last week. Uniquely, YouGov can link its data to how people voted in the 2016 referendum and the 2017 general election – relying not on voters’ sometimes fallible memories, but how they told YouGov they voted at the time of both contests.

Let’s start with all those who voted Labour in 2017. The shift is clear. Defections to remain parties – the Liberal Democrats in particular – rose sharply, while those to leave parties did not. In late April, defections divided two to one in favour of remain parties; by last week, that had risen to three to one. More of those who voted Labour two years ago now plan to switch to one of the remain parties than plan to stay loyal to Labour.

That is not all. If we divide Labour’s 2017 voters into remainers and leavers from 2016, we find – not surprisingly – that the change between the two surveys can be explained completely by those who voted remain. In late April 53% of Labour remainers said they’d now vote Labour, and 45% would vote for another remain party. Last week the number staying with Labour had dropped to 40%, and 57% said they’d look to other remain parties.

As for Labour leave voters, the picture is more intriguing. One-third remain loyal in both surveys, with half defecting to leave parties. But one in 10 Labour leave voters now support remain parties. This is consistent with the findings of other surveys: that a significant minority of Labour leave voters, but not Labour remain voters, are having second thoughts about the wisdom of Brexit.

That said, it remains the case that a fair number of Labour leave voters still want Brexit and plan to switch their support next week, overwhelmingly to the Brexit party. But this should be put in context. In the 2016 referendum, Labour supporters divided two to one in favour of remain. Today the ratio is three to one. This means the number of Labour remain defectors to remain parties is three times as large as Labour leave defectors to leave parties – and has continued to grow.

This relates to a wider point. It has become commonplace to ascribe the leave victory in 2016 to the votes of working-class Labour supporters. This is misleading. Most leave voters live in Conservative constituencies. The Tory shires mattered more than Labour’s industrial heartlands.

A YouGov analysis of more than 25,000 voters suggests the following division of leave voters in the referendum, linked to the 2017 election result.

• Middle-class leave voters: Conservative 5.6 million; Labour 1.6 million.

• Working-class leave voters: Conservative 4.4 million; Labour 2.2 million. (A few of the remaining 3.6 million leave voters supported smaller parties; most did not vote in 2017.)

Why copying the populist right isn’t going to save the left Read more

So the largest block of leave voters were middle-class Conservatives, followed by working-class Conservatives. Just one in eight leave voters was a working-class Labour supporter. To be sure, had even half of these 2.2 million voters backed remain, the result of the referendum would be different. But to suggest that the referendum’s 17.4 million leave voters were dominated by working-class Labour supporters is simply wrong.

This leaves one further issue to be addressed. In this month’s local elections, Labour suffered losses in a number of leave areas. Sunderland, the early harbinger of the leave victory on referendum night, was an example. Does this not indicate that Labour is right to fret about leave voters?

In fact, the story is more complicated. Labour lost 15 seats, and independents three. These 18 seats divided evenly between remain parties (Lib Dem six, Green three) and leave parties (Conservative six, Ukip three). These figures do not suggest that Labour’s problem was confined to pro-Brexit voters.

Besides, although most Sunderland voters backed leave in the referendum, 39% voted remain. Recent polling in the region suggests that remain support has climbed to 45-50%. The data from this month’s council elections does not tell us how the lost Labour voters divide between remain and leave.

None of this addresses the wisdom of Labour’s policy towards Brexit and a new referendum. All it does is indicate that its policy is specifically haemorrhaging remain votes without enhancing its appeal to leave voters. If the party’s aim was to maximise support next week by appealing to both remain and leave Britain, it is failing spectacularly.

• Peter Kellner is a former president of YouGov