At the end of every Commons corridor in the weeks before MPs broke up for the summer recess in 1993, there seemed to be a Conservative whip lurking. John Major's premiership was imploding over Europe as he waged war with a small band of Eurosceptic Tory rebels. The prime minister had poured petrol on the fire by describing the more hardline sceptics in his party as "bonkers" – or, as he put it on another occasion, "three apples short of a picnic". Major knew his opponents were bent on stopping ratification of the Maastricht Treaty, even if it meant destroying his government, so the whips were ordered to get out and about to pull them into line.

For young and idealistic backbenchers, the whips' tactics were a brutal introduction to Westminster. One MP from the 1992 intake recalled last week how they rang members of his family at all hours. "They even kept phoning my wife and saying 'you should tell him to vote with the government'. It was quite extraordinary," he said. "They would try everything – threats and inducements – saying they knew things that they didn't want to have to make public, implying they would if they had to. With some it was affairs, or things like visits to gay nightclubs. It didn't matter if it wasn't true, or was gossip, they still tried it on."

A member of the whips' office during that period recalls how, every day, he and all the other Tory whips would be expected to write notes into a "black book", and that these entries would be discussed each morning at a team meeting in the chief whip's office. "It was mostly fairly mild stuff, about who said what at some meeting of the 1922 Committee, or that kind of thing. But it might be rumours of one sort or another about private things."

For seasoned MPs and ministers it was all run of the mill, just the way things operated. Tristan Garel-Jones, who was a senior whip under Margaret Thatcher and is said to have been the inspiration for Francis Urquhart in Michael Dobbs's House of Cards trilogy, kept the "black book" locked in a safe, and the rumour is that the code to open it was the date of his birthday.

Whips – whether Tory, Labour or Lib Dem – have long inhabited a secret, often dark, world at Westminster, one that no one has ever seriously called into question. But now – just possibly – that could change. When wide-ranging inquiries were announced by home secretary Theresa May on Monday into allegations of child abuse in all corners of public life, including into unsubstantiated claims that a paedophile ring involving political figures operated around parliament in the 1980s, questions about the whips' role as guardians of deep and sometimes sinister information began, at last, to surface.

Labour MP Lisa Nandy was first to draw attention to the possible role of the whips in holding information relevant to the child abuse inquiries. In a Commons debate she referred to comments made by former Tory whip Tim Fortescue, who served in Edward Heath's government between 1970 and 1973. In a 1995 TV documentary, Fortescue, who died in 2008, said: "Anyone with any sense, who was in trouble, would come to the whips and tell them the truth, and say, 'Now I'm in a jam. Can you help?' It might be debt, it might be … a scandal involving small boys, or any kind of scandal in which … a member seemed likely to be mixed up. They'd come and ask if we could help, and if we could, we did."

The late Tory home secretary Willie Whitelaw told the same programme how information was recorded in the black or "dirt" book: "The dirt book is just a little book where you write down various things you know or hear about people that may or may not be true. I think you could make a very good guess what sorts of things it contains."

Fortescue added that "scandalous stories" were whips' stock-in-trade. "When you are trying to persuade a member to vote the way he didn't want to vote on a controversial issue – which is part of your job – it is possible to suggest that perhaps it would not be in his interest if people knew something or other – very mildly."

The BBC, the church and the private school system have already been rocked by child abuse revelations. But now MPs of all parties fear that the scandal could be reaching into their own backyard. Even if there is no truth at all in allegations of a paedophile ring at Westminster, they say that parliament, of all places, cannot be seen to be perpetuating systems under which secrets are recorded, kept and sometimes suppressed.

Defenders of the whips' system say their role as bullies wielding threats is exaggerated and that they are there as much to help MPs who want advice when they hit trouble as they are to twist arms. David Heathcoat-Amory, a Tory deputy chief whip after the 1992 general election, says they are essential to party politics: "If you want a democracy, you have to have political parties, and if you have political parties you have to have discipline, and if you are to have discipline you have to have whips and whips' offices."

But while the whips certainly have a role as counsellors, the information they hold is their real secret weapon and where their power has always resided. Demands are now growing for them to reveal what they know.

Several former Tory whips contacted by the Observer said they never heard stories about MPs abusing children, and if they had, the information would have been passed to police. One did say, however, that rumours had circulated about incidents further back than the late 1980s, and he suspected there may have been "a couple of gruesome cases" before his time.

Tory MP Mark Reckless said on Saturday that it was vital that light was shone into parliament's most secret corners and that it, more than any institution, had to move into the 21st century: "Society has moved on. For a long time there has been a view that this is how things work at Westminster, so this is how things will continue to be. But nobody seems to raise the question of whether it is right or legal."

He added: "I have come into politics believing that MPs should vote according to their consciences, not as part of a nexus of inducements and threats."

Across the board, politicians say Lady Butler-Sloss – whose appointment to head the child abuse inquiry has been widely criticised because of her own close links to the establishment – will have to look where no other inquiry has looked ever before – into the wonderful workings of the whip's offices. Otherwise Westminster will be accused of the ultimate hypocrisy: protecting itself from an inquiry it set up into goings on in our national institutions. There is much scepticism, however. Heathcoat-Amory says: "They have got to try, and they probably will write to ask if there were any written records, but I very much doubt if there is anything there. I think they could be going down a blind alley."

Suspicion about cover-ups and the wholesale destruction of information going back years is mounting. Not only has the Home Office said that a total of 114 files relating to allegations of organised child abuse – including those made by the late Tory MP Geoffrey Dickens that a paedophile ring existed at Westminster in the 1980s – have gone missing. Now further investigations by MPs have revealed that the Conservative whips' office began a systematic policy of shredding all written information it held from late 1996 onwards.

The shredding policy, introduced after a memo to the whips was made public on a judge's orders during Neil Hamilton's unsuccessful libel case against the Guardian over "cash for questions", is revealed in the diaries of former Tory MP Gyles Brandreth.

On 11 December 1996, Brandreth wrote: "The question is what to do in future? The chief's [chief whip's] conclusion is: keep writing notes – he needs the information, so does the PM. But sleep easy, boys: from now on the notes will be shredded on a regular basis."

Senior government figures have confirmed that current and historic records have been destroyed and that the "black book" no longer exists. No documents, no traces.

Reckless has written to David Cameron this weekend asking how the decision to destroy whips' notes came to be made, when it could be argued that such information was government property. Was it part of a cover-up? What bombshells were hidden in those files? He wants the prime minister to compel all Conservative chief whips who have held office since 1960, or their heirs where they are deceased, to provide all the documents that remain in their possession from that time. Some hope.

On Monday home secretary Theresa May told the Commons that the Butler-Sloss inquiry would "have access to all the government papers, reviews and reports that it needs". With so much information shredded or lost in Whitehall, and with a culture of secrecy so ingrained at the heart of power, it looks as if little could be further from the truth.