Matthew Elliott is one of the most successful - and feared - campaigners in British politics. He founded the TaxPayers’ Alliance and Big Brother Watch, and led two victorious referendum groups: NotoAV (the Alternative Vote) and Vote Leave.

But matters have not gone to plan since. In July, the Electoral Commission fined Vote Leave £61,000 and referred it to the police after concluding that the group broke electoral law by collaborating with a separate outfit (BeLeave). Meanwhile, the “hard Brexit” that Elliott and his allies aspired to has been incrementally softened by Theresa May.

“We won a crucial battle but we could lose the overall war,” Elliott, 40, admitted when we met recently at his apartment in Streatham, south London (the second most pro-Remain constituency in the UK). A polite, affable man - who rarely gives in-depth interviews - he explained: “I’m disappointed that the Leave side left the battlefield after the referendum”.

May’s initial Brexit stance and the appointment of cabinet ministers such as David Davis led to a false sense of security, Elliott said. “You can understand why groups like Change Britain scaled down their operations but I still think it was a mistake.”

Elliott himself contemplated maintaining Business for Britain, the outfit from which Vote Leave emerged, but had personal reasons for not doing so. His wife Sarah, an American and the chair of Republicans Overseas, suffered a miscarriage and he wanted to “focus on the family” (the couple now have an 11-month-old daughter).

Though Elliott founded BrexitCentral, a pro-Leave website, he now fears that “we’re moving towards a softer Brexit than we might otherwise have had”. Does he blame May - a Remainer - for this? The Prime Minister triggered Article 50 too early, Elliott replied (“she felt she had to prove herself”). By contrast, “a Leave PM might have been able to explain to the country that, actually, we should hold back and perhaps have more leverage”.

Elliott also suggested that Michael Gove and his combative consigliere Dominic Cummings should have run the Brexit department, rather than Davis and his pro-Remain chief of staff James Chapman (“he had come from the Treasury working with Osborne”). Gove, he said, is “brilliant at pulling a department together, having a clear strategy and a clear message - that would have been a very different kind of DExEU.”

Would Elliott, I asked, like Gove to be the next prime minister? “I’m very wary of entering this line of questioning, I’m not sure an endorsement from me is particularly good for anybody!” But after stating his “admiration” for Boris Johnson, Elliott named new Brexit Secretary Dominic Raab, Home Secretary Sajid Javid and chief secretary to the Treasury Liz Truss as potential leadership contenders. (In October 2017, Elliott was set to become vice chair of the Conservative Party but the move was foiled after news of it leaked and, it transpired, the then Tory chairman, Patrick McLoughlin, had not been told.)

He believes, however, that a pre-Brexit contest would be a “huge distraction” and that the UK will leave the EU next March. “I can’t see any scenario where you go from a position of having Theresa May as prime minister to, all of a sudden, having a new Tory leader and prime minister who, miraculously, has united all factions of the Conservative Party who are then singing from the same hymn sheet - it just won’t happen.”

He added: “It’s really time for people on the Leave side, who want to make sure that the final deal is close to [Theresa May’s] Lancaster House speech, to speak up and make their opinions heard and to get back on the pitch.”

Matthew Elliott was born in Leeds - his father was a social worker and his mother a teacher - and attended the independent Leeds Grammar School before studying government at the London School of Economics, where the self-described classical liberal became president of the Hayek Society and befriended Allister Heath, now the editor of the Sunday Telegraph. In contrast to his Labour-voting, trade unionist father, Elliott joined the Conservative Party “embarrassingly early” at the age of 10-11 (“they addressed the letters to me as ‘Master Matthew Elliott’”).

“I remember at Leeds Grammar School there was an economics teacher called Terry Ellsworth and he basically taught us A-Level economics using Milton Friedman’s Free to Choose book and video series, so I think a lot of my free market economics came from that.”

The Taxpayers’ Alliance, which Elliott went on to found in 2004, is regularly assailed for not publishing its donors. Does he now believe it should? “I don’t think so, I think people have a right to donate to charities and campaigns anonymously.”

Elliott similarly defended the Vote Leave poster which stated that “Turkey - population 76 million - is joining the EU” (something which Gove now regrets). “I would stand by the Turkey message. David Cameron had talked about building a ‘road from Ankara’, it was official government policy, it was official EU policy”.

He added: “I’m proud of the campaign; where my regrets lie is in what happened afterwards. The absence of Vote Leave on the airwaves on the Friday, Saturday, Sunday, Monday, Tuesday, making the case for the sort of Brexit that we’d been campaigning for, I think was really felt. It meant that the airwaves were filled with Nigel Farage and him explaining Brexit in his terms.

“That led to some mistakes along the way: for example, some of the tone around the 2016 Conservative conference, Amber Rudd talking about lists of foreign workers, came from a misunderstanding of the Brexit result and I think that some of the divisions that we now see within the country could have been different had people heard the more positive vision that we were trying to put forward.”

Elliott was also keen to reject the suggestion that he had close relations with the suspected Russian spy, Sergey Nalobin, who was expelled from Britain in 2015, and Conservative Friends of Russia.

“All of this is based on two things: firstly, when [Sarah and I] got engaged and tweeted the picture out, this Sergey guy was the first to tweet back and Sarah being Sarah, loving her social media, responds to all the responses: ‘Thank you so much! That’s so nice.’ And all of a sudden, people think there’s some huge, close relationship there.”

Elliott did attend a Conservative Friends of Russia reception in 2012 and a 10-day trip to the country, but said he had no further involvement. “At the time I was dating a girl who was interested in Russian art and we went along to an event at the embassy, which we’d been invited to by Malcolm Rifkind, of all people, who was president, so very kosher, and I think John Whittingdale was the main speaker, so it all seems very sort of normal. We went along to this reception and then we were invited to go on this trip which I went on - stupid error in hindsight, I probably shouldn’t have gone on it - but, anyway, I did. I attended one reception and I went on this trip.”

Vote Leave has appealed against the Electoral Commission’s ruling and Elliott maintains that he is confident of victory. “The EC based so much of their case on what the whistleblowers were saying, rather than actually listening to us, that when the courts actually go through our evidence, they’ll find in our favour.”

He admitted to being blindsided by the insurgent campaigners (who I interviewed earlier this year). “It did come as quite a surprise when Shahmir [Sanni], Chris Wylie and co. popped up.”

But though he believes that Vote Leave will be vindicated, Elliott conceded that his reputation had been tarnished when I asked whether he would lead a second Brexit referendum campaign.

“First point: I don’t think there should be another referendum. But were there to be one ... If you’re an operator behind the scenes, which I basically am, then if you start becoming the story that’s a bad thing. I would probably become too much of the story and some of the controversies about Vote Leave and the EC would then dog the next campaign. So it would probably be wise not to play a role in the next campaign - before you tempt me to do so.”