As an investigative journalist who has used the Freedom of Information Act to expose the questionable conduct of numerous government departments, I welcome the news that parliament will today consider extending the law. The proposed changes would mean that private companies such as Serco and G4S, as well as housing associations, would have to disclose documents about their vast and lucrative public service contracts.

After countless outsourcing scandals, from the collapse of Carillion to the preventable tragedy at Grenfell Tower, I am sure that many would applaud MPs if they strengthened the FoI. The law is riddled with loopholes, allowing important facts to stay hidden.

While investigating how the Home Office lets private security companies run most of Britain’s immigration detention centres, I repeatedly ran into brick walls. For example, firms such as G4S are obliged to provide monthly “self-audit reports” to the Home Office, detailing what has gone wrong inside the detention centres they are contracted to operate. Any “performance failures” – from a delay in detainees getting seen by a doctor, to fires and escapes, right up to deaths in custody – can incur a financial penalty for the contractor.

The Home Office refused my FoI request to hand over large parts of these crucial reports and censored data on the number of failures because it would “prejudice the commercial interests” of major corporations such as Serco. They said this outweighed the public interest in holding powerful companies accountable. This decision was upheld by a judge – it is the law. Therefore, the proposed changes to that law would open a window into this world of outsourcing, revealing how public money is really spent.

But should parliament go further? Although many of the state’s traditional duties were farmed out to corporations during decades of Thatcherism, New Labour and austerity, the machinery of government has not been hollowed out completely. Throughout the “war on terror”, more money and power was awarded to the so-called “deep state” institutions – Britain’s intelligence agencies and the special forces. None – MI5, MI6, GCHQ or the SAS – are subject to the FoI Act – section 23 exempts documents that even remotely “relates” to them. Leaving aside, for a second, their involvement in secret wars or assassinations, we are not even permitted to have access to their spending on first-class travel or five-star hotels.

It is time for more transparency, without, of course, endangering sources or active missions. We cannot continue to pretend that these state bodies are infallible or immune from being misused by their political masters. We knew little about GCHQ’s snooping programme until whistleblower Edward Snowden sacrificed his career to tell us. We may never have known that MI6 helped render a pregnant woman to Muammar Gaddafi’s dungeons. That only came to light after the dictator fell and incriminating MI6 faxes were found fluttering amid the ruins of Libya’s spy HQ.

‘Many British Sikhs, and indeed the Labour party, are now calling for a public inquiry into British involvement in the Golden Temple massacre.’ Photograph: The India Today Group/Getty Images

It doesn’t have to be this way. We shouldn’t have to wait for an uprising in north Africa, or a rogue Rubik’s Cube, before we find out what is being done, to who and how, in our name. In the US, the CIA is subject to that country’s freedom of information laws. The agency can still refuse to hand over documents if it will harm national security, but they have to prove it in court. If our closest ally can open up its intelligence agency to this kind of scrutiny, why can’t the UK?

This is not just an academic problem. This week, a British judge reached a decision in one of my FoI cases. I had asked the Cabinet Office for several files that could shed more light on British involvement in the Golden Temple massacre at Amritsar in 1984 – when the Indian army killed hundreds of Sikh pilgrims. I wanted to see the files after discovering, through an accidental leak, that Margaret Thatcher sent an SAS officer to advise the Indian army before its disastrous attack on the temple. There were also rumours of MI5 involvement that the government wasn’t ruling out.

After almost four years of legal wrangling, the judge dismissed two of the government’s key arguments, namely that disclosing the files would harm national security or risk trade with India. He also acknowledged that the public interest in this case was “very high”. In the end, though, his hands were tied by the section 23 loophole. The government was not required to hand over the most important file – because it may contain information from the intelligence agencies or special forces.

Many British Sikhs, and indeed the Labour party, are now calling for a public inquiry into British involvement in the Golden Temple massacre. As it stands, only a public inquiry would have the power to disclose enough information on the role played by our intelligence agencies and special forces at Amritsar – but public inquiries can cost millions. It would be far cheaper to reform the FoI, and finally bring these powerful British institutions out of the shadows.

• Phil Miller is a freelance journalist