It was being built to harbour the world's most wanted terrorist, Osama bin Laden. It was built — allegations have now emerged — at the direction of the man sitting down to dinner at Yarralumla: Brigadier Ijaz Shah. The man making the claim is General Ziauddin Butt, the former head of Pakistan's main spy agency, the directorate of Inter-Services Intelligence or ISI. He alleges that Brigadier Shah harboured the world's most wanted criminal for years, at the same time that other arms of the Pakistani military and the US were hunting him. "The most important and all-powerful person in [the] Musharraf regime was Brigadier Ijaz Shah, then Intelligence Bureau chief," General Butt said in a television interview.

"I fully believe that Ijaz Shah had kept this man [Osama bin Laden, in Abbottabad] with the full knowledge of Pervez Musharraf." In a separate interview, General Butt said the Abbottabad compound was built to bin Laden's specifications on Brigadier Shah's orders. But Brigadier Shah told The Age the allegations were groundless. "Not only do I reject it, but any sensible man in the world will reject this allegation. "I retired on March 18, 2008, the Abbottabad incident took place in May 2011 — more than three years later. How is it possible that I am involved in this? The general knows that this is not true."

Brigadier Shah said in more than a year since bin Laden's death, no evidence had been produced that he was harbouring him or knew of his whereabouts. "The whole world has been looking to see who knew about this, but they have not found any proof." But Islamist terrorism expert and author Arif Jamal believes the allegations are credible and that retirement will have made little difference to Brigadier Shah's influence. "Men like Ijaz Shah don't retire in Pakistan, they keep playing different roles for the military," Mr Jamal said. "Ijaz Shah was very powerful when Musharraf was in power, and it makes sense that he knew (of bin Laden's whereabouts). But I don't buy the theory that only the Intelligence Bureau was taking care of Osama bin Laden. Other people in the military were also involved. I think a dozen or so very senior people knew."

Mr Jamal said General Musharraf, still head of the army, almost certainly knew bin Laden was being protected. "I think Musharraf had to have known. The Pakistan army is a very well-ordered army, there is a chain of command, I don't think it would have been possible without his knowledge." Despite his having seized power in a coup, Australia was keen to court General Musharraf when he visited in 2005. Prime Minister Howard signed a counter-terrorism agreement with him promising greater sharing of intelligence. While in Australia, General Musharraf said he believed bin Laden was living somewhere in the tribal belt of the Pakistan-Afghanistan border. "It's very easy for a person to hide," he told the Press Club. "I know that he is alive . . . because of our information and interrogation of various al-Qaeda operatives that we have apprehended."

General Musharraf did not return numerous phonecalls or emails from The Age this week but has said previously he had no knowledge of bin Laden hiding in Pakistan. Somewhat enigmatically however, he has not ruled out others in his regime knowing. "They say Bin Laden was there for five years, that means two years under my watch. Well, one cannot be sure of others, but one can be sure of one's self. And I am 100 per cent sure of myself that I did not know he was there." The Intelligence Bureau and the directorate of Inter-Services Intelligence are two separate and rival arms of Pakistan's massive clandestine security apparatus. The ISI performs the military's espionage, while the IB reports directly to the country's chief executive — in Brigadier Shah's time, President Pervez Musharraf.

In an organisation notorious for walking a fine line between fighting terrorism and fortifying it, Brigadier Shah walked a finer line than most. He was the military's "handler" of Kashmiri terrorist Omar Saeed Sheikh. In 2002, Sheikh kidnapped Wall Street Journal reporter Daniel Pearl and handed the US-Israeli citizen over to other militants, who beheaded him. Days later, Sheikh handed himself into Brigadier Shah, who held him for a week in a safehouse before finally handing him over to authorities, allegedly to give Pearl's murderers a chance to escape. Before her own assassination, former Pakistani prime minister Benazir Bhutto alleged Brigadier Shah was conspiring with terrorists to assassinate her, naming him in a letter as someone who should be investigated if she died.

Brigadier Shah was in charge of Bhutto's security detail, which failed to protect her, on the night she was killed in 2007. Brigadier Shah, now in his mid-60s, is a man with strong links with Australia. His children studied in Melbourne and Sydney, and, in the weeks after Bhutto's death he reportedly left Pakistan for Australia for several months. In 2004, then-president Musharraf nominated him to be High Commissioner to Australia. He was rejected by Canberra, it is understood, out of concern over his links to terrorists. However, Brigadier Shah was a prominent member of Musharraf's entourage on a state visit to Australia in June 2005. He sat down to dinner at Government House on the night of June 14.

At that time, construction of the compound that was to house bin Laden, his wives and acolytes had been underway for at least six months. Bin Laden may have already moved in — one of his wives, who survived the attack that killed him, said they had been living there since 2005. Brigadier Shah said it was impossible that knowledge of bin Laden's whereabouts could be limited to a handful of people. "There is no way that one man, or even one institution, can keep this man hidden from all the entire world, this man that the entire world is looking for." Last month, Brigadier Shah was interviewed in camera by a Pakistani government inquiry charged with investigating how bin Laden was able to hide in Pakistan for so long. "People should wait for that committee's report, it is run by three very eminent people and has gathered all of the information about the Abbottabad operation. But I knew nothing about that, I was never involved in anything like this [harbouring bin Laden], absolutely not. This is propaganda people are saying for their own interest."

In the treacherous world of Pakistani military politics, General Butt is a known enemy of General Musharraf, who sacked him as chief of army staff after only hours in the job and jailed him for two years. Brigadier Shah told The Age that General Butt was part of a concerted campaign to discredit the exiled former leader and his allies. "The General is settling old scores," he said, "he is trying to damage people by alleging everything about everyone. This is the way in Pakistan." But the global consensus is that bin Laden was protected by senior officials in the Pakistani military or government. The White House's counter-terrorism chief John Brennan said it was "inconceivable that bin Laden did not have a support system in the country", while Defence Secretary Leon Panetta said he believed somebody in authority in Pakistan knew of his whereabouts.

And an email reportedly sent from security affairs think tank Stratfor on May 13, 2011 — just days after the raid on the Abbottabad compound which killed bin Laden — and later published by WikiLeaks said at least 12 people knew. "Mid to senior level ISI and Pak Mil[itary] with one retired Pak Mil General ... had knowledge of the OBL arrangements and safe house", Stratfor's vice-president of intelligence, Fred Burton, wrote to colleagues. Follow Ben Doherty on Twitter @SouthAsiaCorro