Special Minister of State Mal Brough has confirmed he has handed over documents relating to the disclosure of former speaker Peter Slipper's diary after he was visited by Australian Federal Police officers at his home on Tuesday.

Labor MP Graham Perrett, who had written to the AFP about Mr Brough's role in the case, has questioned whether the Queensland MP should remain in the ministry while the matter is under investigation.

"The Westminster system doesn't disintegrate if a minister steps down while they're being investigated. If you look back at the Westminster tradition, ministers have stood aside for less," Mr Perrett told Fairfax Media.

In a short statement on Thursday, Mr Brough, who is also the Minister for Defence Materiel and Science said: "I can confirm reports that the AFP visited me on Tuesday requesting any documentation relating to allegations involving the disclosure of diary notes of Mr Slipper."