As Ukip continues to rise up the polls, the political establishment is sinking to even greater depths to smear us. The latest attack comes in the form of Barbara Roche, the former Labour immigration minister, who is setting up a cross-party group aimed at calling Ukip’s election campaign and, by implication, everyone who supports it racist. It shows an astonishing lack of awareness of what voters are concerned about, in my opinion, and yet I am not surprised.

As I’ve been travelling around the country on my tour ahead of the European Elections, one thing has been consistent, aside from the packed venues: the rabble of left-wing protesters outside any venue, trying to stop people from taking part in the political process.

These protests are mainly led by Unite Against Fascism and Hope Not Hate, the latter an organisation whose leader is diametrically opposed to the libertarian-leaning Ukip. Nick Lowles is associated with the Communist Party and the Socialist Workers Party, and like the unions who give large donations to their organisation, is mightily upset that Labour voters are coming across to Ukip.

Download the new Independent Premium app Sharing the full story, not just the headlines

While we don’t yet know about the funding for this cross-party group, we do know about funding for Hope Not Hate. For since 2010 it has received £150,000 in funding from the Department of Communities and Local Government, ostensibly to campaign against “racism in deprived areas”. In 2012 it received no less than £63,000 of taxpayers’ money, which given that the group is so open in its hatred of Ukip seems to me an entirely unsuitable use of much-needed government money.

But if we have once again retreated to the times when even mentioning immigration results in accusations of extremism, then is Barbara Roche also going to condemn fellow Labour MP Frank Field? It was only just over a year ago that Mr Field, writing in The Daily Telegraph alongside Conservative MP Nicholas Soames, said that Ukip’s success in the Eastleigh by-election had made the struggle to debate the subject much easier.

“For nearly five years the Cross-Party Group on Balanced Migration, which we co-chair, has been pointing out the consequences of large-scale immigration,” Field and Soames wrote. “We have called for action to reduce it. But, though many of our recommendations have been accepted, the campaign has always been an uphill struggle.”

A look at the immigration policies of the four main parties show that actually, Ukip’s policy is the most open towards non-white migration. For despite the howls of “racist” and now “Euracist” which are frequently thrown at our supporters and activitists, we are anti-uncontrolled immigration. The only way to end this is to leave the EU and introduce a points-based visa system based on merit and need.

Wanting to control immigration is something which appears in all the party manifestos. The Conservatives have been vociferous in vocalising the need to “control immigration” so that “we only welcome those who want to work hard and contribute to our society”. Their cap on net immigration, however, while remaining part of the EU, means that the people they discriminate against come from the rest of the world, where people tend not to be white.

Even the pro-immigration Lib Dems have realised that uncontrolled immigration is not popular with the majority of voters and that people are feeling the impact of over-supply in the unskilled Labour market. They too call for “more controls to the system” and boast of the apparent success in helping to cut immigration while they have been in coalition.

But back to Ms Roche’s own party, Labour. You would think given the establishment of this group that Labour trumpet their policies of the late 1990s. But Frank Field is not alone when it comes to championing controls and realising that the reason Labour is losing support to Ukip is because the working class have been hardest hit by Labour’s disastrous policies. Only a few weeks ago the shadow Home Secretary Yvette Cooper said that “action is needed to tackle the unfair impact of immigration” and that “stronger controls are needed”.

So what is it about Ukip that makes it racist for us to talk about migration, but not the other parties? Could it be because we’re not part of their club? It smacks of a cartel, trying to restrict entry into the market for new competitors. And I think it’s one which will backfire on them.