Tory MP Tim Loughton faced a barrage of criticism on Monday after suggesting that the Liberal Democrat Sarah Teather had been a poor families minister because she failed to "produce" a child of her own.

In remarks which were angrily condemned by Liberal Democrats, the former children's minister claimed Teather had made the Department for Education (DfE) a "family-free zone" when they worked together. Both lost their ministerial jobs in the 2012 reshuffle.

David Heath, the Lib Dem agriculture minister, told the Guardian that Loughton's unhappiness at losing a government job had coloured his judgment, saying: "We get it by now - Tim Loughton is bitter about being sacked. But that comment is inappropriate from anybody."

Loughton's remarks about his time working with Teather in government were made at a Tory event in Windsor.

"The person who was actually in charge of family policy among the ministerial team at the DfE was Sarah Teather. Which was a bit difficult because she doesn't really believe in family," he said.

"She certainly didn't produce one of her own. So it became a bit of a family-free zone. I think that is a huge disappointment."

Addressing Conservative activists at the weekend, he said his support for policies such as a married couples' tax break had been frustrated by Teather.

Asked about his criticisms by the Telegraph, Loughton said it was a "throwaway comment" and "not really fair". He said he wanted to focus on the fact the Lib Dems oppose rewarding marriage in the tax system because they have "little interest in family policy and certainly not traditional views of what constitutes a family".

Later, he said on Twitter that he would say sorry to Teather. "I will of course apologise to Sarah for the comments that have been reported and were certainly not intended or indeed said in that way," he wrote.

Loughton told the meeting the decline in the numbers of children born to married parents had contributed to the "chaos of the herd instinct" in the 2011 riots, and said the Conservative party was failing its own supporters by doing too little to support marriage.

"Many hard-working traditional families have been facing some pretty tough economic times, some pretty tough losses of benefits and tax-breaks at our hands, and these by and large are our supporters and our people. We have failed them to a large degree," he said.

Loughton, who is married and has three children, has been a persistent critic of Michael Gove's department since leaving office, saying it was run like the fictional department store Grace Brothers.

He added he would continue to push the coalition to support married couples from the backbenches.

Teather declined to comment. She will not seek re-election in 2015, she revealed this month.

Kate Green, the shadow equalities minister, said Loughton's comments were inappropriate.

"It's particularly distasteful to talk about people's private family choices and personal circumstances," she said.