Nigel Farage will insist that Ukip voters are "mostly in the middle" of the political spectrum as he seeks to brush off allegations of "shouting Hitler Youth songs" as a schoolboy.

At the party's conference on Friday, the Ukip leader will challenge David Cameron to reject uncontrolled immigration from Bulgaria and Romania or face the wrath of voters in next year's European elections.

But he faces a fresh row before Ukip's annual gathering in London after Channel 4 News obtained a letter written 30 years ago by a teacher at Farage's old school, Dulwich College. The letter to the head of the school, written by Chloe Deakin, expressed concern that Farage had been made a prefect despite reports of "publicly professed racist and neo-fascist views".

Dated 4 June 1981, the letter says one of Farage's teachers described how the schoolboy and others "marched through a quiet Sussex village very late at night shouting Hitler Youth songs" while he was in the cadet force. Farage said any suggestion of singing the songs was "complete baloney", but admitted: "Of course I said some ridiculous things – not necessarily racist things." He told Channel 4: "It depends how you define it. You've got to remember that ever since 1968 up until the last couple of years, we've not been able in this country intelligently to discuss immigration, to discuss integration. It's all been a buried subject – and that's happened through academia, it's happened through politics and the media."

The news comes after Ukip has taken action against several members for controversial remarks on social media and a row over MEP Godfrey Bloom referring to foreign aid recipients from "bongo-bongo land". In his conference speech, Farage will say the party opposes all racism but defend the right of members to make "pronouncements that I would not always choose myself".

The Ukip leader will also lay down a challenge to Cameron to reject "unconditional" immigration from new EU countries by the start of next year. He will claim a "Romanian crime wave" in London shows a "darker side to the opening of the door in January".

In an appeal to first-time voters, he will say next year's European elections are a chance for people to express their desire to leave the EU "without worrying which lot will get into Downing Street". People should treat it like an EU referendum, he will say, making a vote for Ukip a vote for the exit door from Brussels.

Farage also is expected to characterise typical Ukip voters as coming from a "range British society from all parts of the spectrum" who feel "fed up to the back teeth with the cardboard cutout careerists in Westminster.