There was no luxurious suite for Vince Cable when he checked into the Ramada Plaza in Southport for his party’s Spring conference, but a cramped standard double that he shared with his wife, Rachel. If this was a man on his political uppers – on just nine per cent of the vote, according to the latest Survation poll – he presented good reason to be quietly confident when I squeezed in to see him yesterday.

“I have a historical perspective and I remember we were on about that percentage when we went into the 1997 election, which turned out to be a breakthrough year for us,” he tells me. “We surged to 16 per cent and doubled our number of MPs and became a major force in Parliament. Obviously, I would like us to be further ahead in the polls now, but I don’t panic about opinion polls and my sense is genuinely that things are beginning to move our way.”

In his typically unshowy, under-stated way, Cable had minutes earlier made the kind of political speech you all too seldom hear these days: it was idealistic. The BBC may have run a headline saying there was a “backlash” against him over what he had said about racism being a factor in the vote for Brexit – this “backlash” came, unstartlingly, from Tory MPs – but it struck a chord and won him a long standing ovation in the conference hall.

Cable’s talk of a “breakthrough” year may well not be wishful thinking. Over the past few weeks, there has been feverish gossip in Westminster about high level defections to his party and even a fundamental realignment of British politics. “I get individuals from both the Tory and Labour benches coming up to me in the Commons, maybe after a late vote when it is possible to talk discreetly, to tell me they feel their positions in their respective parties are completely untenable. I take that to be coded language, but I am not pushing it and I am happy to leave it to them to come on their own terms and in their own time.”

He believes the point of departure for them – to use the title of the late Robin Cook’s book – is most likely to come in the autumn, when Theresa May will ask them to either accept or reject the deal she has come up with to leave the European Union. “It is possible if some Tory MPs vote against their party line their position could be made impossible and they will find themselves in limbo within their party.

“I think for members of the Labour Party it is a different matter. For the moment, Corbyn is being given the benefit of the doubt. They had a reasonable election – they didn’t win, of course, even if they sometimes give the impression they did – but there are a great many Labour MPs I know to be desperately worried about where their party is going in the longer term and they may well jump or be pushed. Obviously I don’t know precisely who or what or how.”

Cable chooses his words with care, but the names in the frame have included Chuka Umunna and Anna Soubry. “I know both of them reasonably well and they would be welcomed with open arms if they did choose to join us. They are, however, instinctively loyal to their own people and I respect that. I’m not waiting with membership cards for them, but if they come I would be delighted.”

Although he is aware that such speculation excites journalists, he has more immediate concerns. Before the Summer recess, he says his party is co-ordinating the efforts in the House of Lords to get through amendments toning down some of the more extreme aspects of the Brexit legislation, which he anticipates will be passed with big majorities.

“The more important issue is the customs union, and, given that Labour have changed their position a bit, that provides an opportunity for a Tory rebellion in the Commons. We are promised one. I am not holding my breath, but it could happen. If it did it would give Mrs May the sense that she is on the skids.

“After the Summer, we will have, of course, what is called the meaningful vote. I think that, too, will be important, but not decisive. It is certainly important that MPs from all sides of the House heed John Major’s advice and do what's right. Even more important for us will be getting agreement that there will be a popular vote on the final deal. Without that, there will be a danger of ill will for generations. We need a popular vote to unite the country.”

Cable is aware of, and welcomes, the legal action being mounted by Best for Britain to try to compel Mrs May to hold a second referendum. “I think it is quite useful to have these legal challenges. We saw how Gina Miller completely changed the weather by the case she brought against the government so I think it is helpful to see that they act within the law and the constitution. I would say though that at the end of the day it will have to be for parliament, rather than the courts, to decide if we as a country drop Article 50. Similarly, having a popular vote on the final deal would be a decision of Parliament, rather than the courts’.”

In the meantime, scarcely a week goes by without a new party being set up to challenge Brexit – something that Cable clearly finds tiring. “The fact there is a multiplicity of them says it all. They obviously don’t understand how the British voting system works. They are mostly, so far as I can see, driven by one or two rich individuals who are doing it as some kind of vanity project. They are not serious. It is not helpful as instead of having a highly focused single united force a lot of energy is being dissipated. I think there is more coherence among the various campaigning groups, but, again, I would rather we all came together under one umbrella as we don’t need what I call the Narcissism of small differences. Still, there is a co-ordinating group I sit on with Anna Soubry, Chuka, Dominic Grieve and Chris Leslie, among others, and they are a good group and they are workmanlike so we have at least got to that stage.”

For a man of 74, Cable has extraordinary energy – he was back in London and in the Today studio for just after 7 this morning to he hectored by Humphrys over the “white faces” line in his speech – and he remains at heart optimistic that Brexit will soon be consigned to the ashcan of history. “The polling on that shows people are coming around to our way of thinking, slowly but surely. I think it is important people do believe it can be stopped because the biggest problem we have now is the slightly fatalistic view that nothing can be done. It can be done and it will be done, believe me, because Brexit is simply wrong.”