The National Party has issued a letter expressing "serious concerns" over Speaker Trevor Mallard - including over his chairing of the House and his alleged role in "negative stories."

The letter, written by Shadow Leader of the House Gerry Brownlee, alleges the Speaker had a role in Newshub's story about a National Party MP calling this Prime Minister a "stupid little girl".

It's almost tradition to test the Speaker in the House, but a letter demanding a "full explanation" about the Speaker's "role in pushing this story" is far from common.

"Promoting stories about events which cannot be verified and which reflect badly on the Opposition is unacceptable from the Speaker," the letter says.

The letter says National's confidence in the Speaker has been "significantly shaken".

Last week, Newshub reported a male National Party MP called the Prime Minister a "stupid little girl" during Question Time the previous week.

At the time, Mr Mallard stopped proceedings and asked the person who made the remark to stand, withdraw and apologise. No one owned up to the comment, and all 10 male MPs who sit near Simon Bridges later told Newshub they didn't make the comment and didn't know who did.

Stuff reports Mr Mallard believes he heard a National MP call the Prime Minister a "stupid little girl" but that Mr Bridges believes there is "no evidence" the remark was actually made.

Mr Brownlee's letter also accuses the Speaker of arbitrarily taking questions off the Opposition during Question Time.

Deputy Leader of the Opposition Paula Bennett exited the House during Question Time on Wednesday, voicing "frustration" over the question penalty.

It was a particularly eventful afternoon in the House, with the Speaker calling out another sexist comment, this time from a Minister of Government. New Zealand First's Ron Mark told presumably Amy Adams to "settle petal" during debate. He stood, withdrew the comment and apologised.