Howard Flight who helped to raise millions for Tories, suggested welfare changes would encourage poor to have more children

David Cameron was under pressure tonight to evict a new Tory peer from the party after he suggested that welfare changes would encourage "breeding" among the less well-off.

Labour and the Liberal Democrats denounced the "shameful" remarks by Howard Flight, who was rewarded with a peerage by the prime minister last week after helping to raise millions for the Conservative party in the runup to the election.

Downing Street was forced to distance itself from a second Tory peer in a week after Flight warned that plans to remove child benefit from higher-rate taxpayers would deter the middle classes from having children.

His remarks followed the claim last week by a former Thatcherite cabinet minister that most people were better off in the recession. Lord Young of Graffham was forced to resign as Cameron's enterprise adviser after suggesting that most voters had never had it so good as during the "so-called recession".

Flight told today's London Evening Standard: "We're going to have a system where the middle classes are discouraged from breeding because it's jolly expensive … But for those on benefits, there is every incentive. Well, that's not very sensible."

This is the second time Flight, a former Tory deputy chairman, has fallen foul of the party leadership. He was barred from standing as a parliamentary candidate by Michael Howard in 2005 after suggesting that Tory spending cuts did not go far enough.

Douglas Alexander, the shadow work and pensions secretary, said of Flight's interview: "These shameful but revealing comments cast serious doubt over David Cameron's judgment in personally appointing Howard Flight to the House of Lords only a few days ago. Last week one of the prime minister's senior advisers told us we'd never had it so good, and now his latest hand-picked peer comes out with these comments. Instead of dithering for hours, as he did with Lord Young, David Cameron should take swift action."

Flight initially claimed that his remarks had been taken out of context. But he later issued an unreserved apology and withdrew them after Downing Street warned him that they were totally unacceptable. "I apologise unreservedly for any offence caused and would like to withdraw the remarks," Flight said in a statement issued by the Conservative party.

Cameron refused to remove Flight from the House of Lords, though some Tories said that would be possible because he has yet to take his seat. "I don't agree with what he said and I am sure that he will want to apologise for what he has said, and I think we can probably leave it at that," the prime minister said at a Downing Street press conference with his Swedish counterpart, Fredrik Reinfeldt.

The remarks by Flight, an outspoken figure popular in Tory circles, echo an infamous warning 36 years ago by the late Sir Keith Joseph, Margaret Thatcher's intellectual guru. Joseph said in 1974: "The balance of our population, our human stock, is threatened." A recent study, Joseph added, had shown "a high and rising proportion of children are being born to mothers least fitted to bring children into the world and bring them up".

Flight was appointed to the Lords as a reward for his role as president and founder of the Conservative City Circle, which he set up to improve links with the City and to encourage donations to the party. Cameron also thought that Flight was treated unfairly by Michael Howard, who removed the Tory whip from him in the runup to the 2005 general election. This meant that Flight was disqualified from defending his Arundel and South Downs seat as the Tory candidate.

The former shadow Treasury minister was also critical in his Evening Standard interview of the government's plans to raise university tuition fees. The millionaire financier said: "Two of my nieces and nephews, both of them very bright, gave up university halfway through because they didn't want the financial burden."

Brendan Barber, the TUC general secretary, said: "Howard Flight has shown himself to be an insensitive throwback to the worst of 1980s politics within days of being made a peer by the prime minister."

Bob Russell, the Lib Dem MP, said: "His comments are offensive and unacceptable. They are not appropriate for the 21st century, especially when the gap between rich and poor is growing and when the poorest still have lower life expectancies."