Conservative Leader Andrew Scheer has called on Finance Minister Bill Morneau to apologize for using the term “Neanderthal” in a heated conversation with Conservative MP Lisa Raitt during Monday’s House finance committee meeting.

“It was quite clear he had lost his temper and he was agitated. Now it’s time for him to apologize,” said Scheer.

Scheer went on to say that if Morneau won’t deliver the apology himself, the prime minister should order him to do so.

Morneau’s puported misstep came about when Raitt grilled him on the underrepresentation of women at the executive level at Morneau Shepell — the finance minister’s former company — as well as among Morneau’s ministerial staff and at Finance Canada. Morneau asserted the government’s commitment to promoting women to leadership roles and, in so doing, made the controversial comment.

“We’ll continue to (promote women into positions of leadership). If people like you don’t buy into it, that’s a problem that we’ll have to face,” Morneau said. “My view is that we will be more successful collectively if we’re actually able to successfully promote women into leadership roles. We will drag along the Neanderthals who don’t agree with that, and that will be our continuing approach.”

“I’m not a Neanderthal,” Raitt retorted.

Morneau defended himself in a statement from his office, which refers to Scheer’s move as “smear tactics” and “offensive.”

“The exchange was clearly not meant to single out one Member of Parliament, but rather to illustrate how backwards the entire Conservative Party‎ has been on promoting gender equality,” the statement said.

Scheer didn’t buy it. He argued that in the footage of the conversation, Morneau is “speaking directly at Lisa Raitt” and is “telling her ‘people like you.'”

“If his excuse is that he was using inappropriate language to describe everyone, I don’t know if that holds much water,” said Scheer.

The Conservative leader also likened the situation to former Conservative MP Gerry Ritz’s “Climate Barbie” remark directed at Environment Minister Catherine McKenna. Scheer pointed out that in that case, he “did the right thing” and called McKenna to apologize. He said Ritz apologized as well.

“I’m calling on Justin Trudeau to order his finance minister to apologize for those outrageous, insensitive, and sexist remarks,” Scheer said.