A senior Ukip politician who said Britain should not be sending aid to "bongo bongo land" has been ordered by his party to stop using the phrase because it could be considered "disparaging" by foreigners.

Ukip issued the rebuke to Godfrey Bloom, a member of the European parliament, after his remarks made at a meeting of activists sparked outrage among some MPs and anti-racism campaigners.

Bloom took to the radio on Wednesday morning to defend the use of the phrase "bongo bongo land", saying he was just articulating the views of "ordinary people in the rugby and cricket club".

But shortly afterwards, the Ukip leadership distanced itself from Bloom's expression.

Steve Crowther, the Ukip chairman, said: "We are asking Godfrey not to use this phrase again, as it might be considered disparaging by members from other countries. However, foreign aid is an extremely important debate that needs wider discussion."

Bloom made the comments to a meeting of supporters in the West Midlands, suggesting foreigners used aid to "buy Ray-Ban sunglasses" and "apartments in Paris".

His remarks have emerged in the week Ukip is due to publish its list of approved candidates for next year's European elections, in which the party hopes to get the biggest share of the vote.

Bloom was filmed speaking at a meeting in Wordsley, near Stourbridge, in July. In the recording he says: "How we can possibly be giving a billion pounds a month when we're in this sort of debt to bongo bongo land is completely beyond me.

"To buy Ray-Ban sunglasses, apartments in Paris, Ferraris and all the rest of it that goes with most of the foreign aid. F18s for Pakistan. We need a new squadron of F18s. Who's got the squadrons? Pakistan, where we send the money."

Speaking on Today, he said he did not see why the government should send money overseas when hospitals were being closed down and British people were unable to receive cancer treatment.

"If I want to send money to charity, I will do so of my own accord," he said.

Asked whether he would apologise if instructed to do so by the Ukip leadership, he said: "I'd say right-o, sorry. If I have offended anybody in bongo bongo land, I will write to their ambassador at the court of St James."

Bloom told the programme he was merely articulating the unheeded views of ordinary people, adding: "My job is to upset the BBC and the Guardian and I love it."

Two months ago Farage ensured an Italian MEP was expelled from Ukip's European alliance for saying a black minister in Italy was part of a "government of bongo bongo" who would want to impose "tribal traditions", and would be better suited as a housekeeper.

Bloom said suggestions that his comments carried any racist implications were "absurd" and "laughable", adding that he had two Kashmiri staff and a Polish wife. He stood by his remarks, saying: "What's wrong with that? I'm not a wishy-washy Tory. I don't do political correctness … The fact that the Guardian is reporting this will probably double my vote in the north of England."

In another part of the recording, Bloom says, in reference to a ruling from the European court of human rights: "You can torture people to death but you jolly well can't give them a full life sentence because that's against their human rights.

"We can't hang them because we're now a member of the European Union and it's embedded in the treaty of Rome.

"It's a personal thing but I'd hang the bastards myself … Especially for some of these, especially for the guy who hacked the soldier to death. I do hope they would ask me to throw the rope over the beam because I'd be delighted to do so."

There have been reports that Ukip has been seeking a higher level of vetting for its prospective MEPs, after Farage admitted a "handful" of its local election candidates had caused the party embarrassment.

Bloom's remarks caused outrage among many MPs and campaigners. Rushanara Ali, a shadow development minister, said: "It's just offensive and the kind of thing that should have been consigned to the history books. It's completely at odds with the 21st century.

"If Nigel Farage is serious about getting rid of racism and intolerance in his party, he should take action against politicians who think it's acceptable to speak of people in developing countries in that way."

John Mann, Labour MP for Bassetlaw, said Ukip should "throw him out and stop him standing as an MEP". A spokesman for the Hope, Not Hate campaign said Bloom's remarks were reminiscent of the "Tory party of 1985", when Alan Clark provoked outrage by referring to Africa as "bongo bongo land" in an official meeting.

Bloom has previously caused controversy over some of his comments about women. The MEP was criticised for asking why businesses would ever hire "a lady of child-bearing age" and once said he wanted to get involved in women's rights issues because "I just don't think they clean behind the fridge enough".