The Conservative defence minister, Anna Soubry, has apologised to the Ukip leader, Nigel Farage, after suggesting that his facial expressions made him look like someone who enjoyed a person putting their finger up his bottom.

Soubry made her remarks on the BBC's Andrew Marr Show while sitting alongside the impressionist Rory Bremner, Lord Mandelson, and the business secretary, Vince Cable. Both politicians looked on astonished as presenter Andrew Marr tried to stop her.

After Bremner had impersonated Farage, Soubry – who has clashed with the Ukip leader before – said: "I always think he looks like somebody has put their finger up his bottom and he really rather likes it." Bremner asked: "Are you allowed to say that?", at which point Marr said: "It's too late. She has said it." Lord Mandelson commented: "Anna, please, it's too early."

The comment provoked an immediate reaction on Twitter, not least from Farage himself. "Perhaps [Anna Soubry] should spend less time investigating digital rectal insertion and more on her brief?" he tweeted.

Farage added: "She is rude and incompetent. I would expect an apology – but I won't be holding my breath."

In a statement, he said: "This is a woman who was proven to not even have a grip on her portfolio back in November when on Question Time she showed she had absolutely no idea about the Royal Navy support ships being built in South Korea.

"She also launched a vile attack on me then, claiming I was 'putting fear into people's hearts' by raising concerns over unrestricted immigration. The same concerns that today her party are so desperate to try to address.

"But this latest, remarkable foul-mouthed attack is utterly incredulous. Soubry has stooped to the levels of crudity that any politician would spend a lifetime apologising for.

"This is supposed to be a professional woman representing her political party on a national television programme on a Sunday morning. Well, I am astonished, and I would imagine the rest of her party is too, as well as deeply embarrassed."

With Downing Street unimpressed by the language and insults, Soubry later apologised. "If he is in any way offended, I apologise unreservedly. It was a lighthearted remark," she said.

The pair's previous run-in came last month, when Soubry said on BBC Question Time that Ukip had published a leaflet in her constituency suggesting that 29 million people would be coming to the UK from Romania and Bulgaria, The joint population of the countries is only 28 million.

Soubry told Farage: "I do not like your tone. You don't talk facts, you talk prejudice, you scaremonger, you put fear in people's hearts." She said Farage was trying to turn foreigners into a menace, adding, with reference to the 1930s, that history had taught Britain the danger of such an approach.