Labour and the Tories are locked in a duel over Brexit that is like one of those arcane cycle races where the two riders start as slowly as possible. Just as we have returned to two-party politics, with Labour and Conservatives continuing neck and neck at around 40% in the polls, both appear to contain more fragile coalitions than at any point in recent history. No one wants to make the first move for fear it will hammer a wedge into the fracture in their parties.

Both parties declare that they respect the democratic decision of the EU referendum. But in the end they probably pay more attention to the electoral maths. Around two-thirds of Tory MPs voted to remain in the EU, but more than 60% of the party’s members and supporters voted to leave. An even larger proportion of Labour MPs, around 95%, are remainers, while more than 60% of party members and current supporters back remain. But the MPs are very aware that two-thirds of Labour constituencies voted leave and more than 60% of their traditional support base – C2DE working-class voters – backed leaving.

But as Michel Barnier, the EU’s chief negotiator, has pointed out, we are nearing the point where firm proposals have to be put forward. The most immediate crunch point is the question of the customs union – a tariff-free area that allows goods to cross borders painlessly. With a bespoke “customs partnership” based on virtual borders little more than a figment of some speechwriter’s imagination, the divide within the government over the issue looks to be unbridgeable.

It’s a policy that could bring together a large part of the country, whether they voted leave or remain

Labour’s official position at the moment is also that the UK will leave both the customs union and the single market, replacing them with participation in “a customs union” and “a single market” ... angels dancing on the heads of pins. The party now says it will review the position on the customs union, though the leadership of Jeremy Corbyn and John McDonnell remains wary of a backlash among its northern and Midlands voters should there be a change. But are these fears justified?

Like 40% or so of Labour voters, I voted to leave, fed up with domestic austerity, EU rules on government deficit and national debt ratios, and the EU’s decade of mass unemployment to shore up the euro and the banks. Many hard remainers in Labour put staying in the EU above all other considerations. But many Labour people will still put dealing with Britain’s inequality, growing poverty and the marginalisation of so many people in the old industrial areas (which mostly voted leave) as their top priority. And that means toppling this paralysed Tory government and electing a Labour government that will enact the radical policies to transform this situation, whether or not they clash with what the EU would prefer.

That’s easier said than done, but maybe now is the time to be brave and put the Tories on the spot by backing staying in the EU customs union. That has several pluses. It sorts out the Irish border problem and avoids huge tailbacks at Dover and other ports; there need be no disruption of supply chains for the car and other industries; and it spikes the neoliberals’ Singapore option – a deregulated, free-trading billionaires’ playground (which is why the Tory hard Brexiters are so opposed).

It’s also a policy that could bring together a large part of the country, whether they voted leave or remain. In 1975 Britain voted two to one to stay in the European Economic Community – essentially a customs union. It’s the over-regulation required by the single market and the political ambitions of the EU after Maastricht that rile people. Of course that would mean being firmer about leaving an unreformed single market. But the policy of the Labour hard remainers of staying in the EU while pretending we haven’t – staying in the single market, with free movement, large budget contributions and European court jurisdiction – fools no one.

Up to now, the Tories have been able to sideline rebellions from their own remain contingent (Ken Clarke, Anna Soubry, Dominic Grieve and so on) by abstaining on Labour motions where defeat looked likely, such as on a final parliamentary vote on any deal. But it’s hard to see how they could abstain on such a major issue as the customs union. And while the DUP MPs who prop up the Tories are leavers, above all else they do not want a physical border in Ireland, or between Northern Ireland and the rest of the UK.

The Labour leadership has been right to hedge its bets and let Tory divisions fester when it has no role in the negotiations with the EU. But it too has to make up its mind on the future direction. It could declare its support for staying in the customs union, but back it up by reaching out (as it has so far failed to do) to Walsall and Stoke and Mansfield, and all the other deprived leave-voting areas, with a new-deal plan of investment in infrastructure, training and jobs. Now may be the time for Labour to take a risk, strike out and leave the Tories floundering.

• Nigel Willmott is the Guardian’s letters editor