The revelation that UK universities have spent around £87m on payoffs to staff that come with “gagging orders” in the past two years has again highlighted concerns that such secretive clauses are being used to conceal the extent of harassment and bullying at higher education institutions.

UK universities pay out £90m on staff 'gagging orders' in past two years Read more

Dozens of academics told BBC News they were “harassed” out of their jobs and forced to sign non-disclosure agreements (NDAs) by their university after making complaints. Although the BBC admits it does not know how many of these gagging clauses relate to sexual misconduct or other kinds of harassment or bullying, the findings highlight a wider lack of transparency in the way universities handle all kinds of misconduct and discrimination.

When the Guardian sent freedom of information requests to more than 120 UK universities in 2016 about the scale of sexual harassment by university staff, only three institutions confirmed that they had used NDAs or confidentiality clauses in settlements for complaints. However, since then I have become aware of three more universities who used such clauses in sexual misconduct cases. Whether those institutions deliberately misled me is hard to prove. Anecdotally, I’ve been told that some universities’ legal departments have determined that NDAs trump their duties under freedom of information law to reveal the existence of such gagging clauses. If such legal advice is widespread, then the information commissioner needs to step in.

The BBC’s investigation found that many universities do not record the reasons why a gagging clause was used. It is likely that many were used in unrelated redundancy and severance deals, as this has become standard practice in many industries, not just higher education. However, BBC reporter Rianna Croxford told me that “some people … said they had to ‘choose’ what their NDA was over ie employment, harassment, or racism and this affects how it’s recorded”. This is consistent with my own investigations, where sexual harassment victims said their complaints had been reclassified as being about “poor performance” to avoid the risk of reputational damage to the university, and make it easier for perpetrators to find work elsewhere. In other words, this made it easier for the university to pass the buck on to another institution.

The sexual misconduct scandal at Goldsmiths, University of London illustrates just how damaging this secrecy can be. The Guardian was told the names of several male members of staff who were allegedly the subject of inquiries at Goldsmiths between 2013 and 2014 following complaints from students about harassment and sexual misconduct. Settlements were made and some staff left, but many of the details could not be reported because parties involved signed a confidentiality agreement. Furthermore, students wanted to know whether any of the alleged perpetrators were still at the university, and, if not, whether other institutions were aware of the claims against them. Only when Prof Sara Ahmed, who resigned from her post at the university in the summer of 2016, blew the whistle did the scandal emerge.

Astrophysicist Emma Chapman, one of the harassment complainants interviewed by the BBC, and other campaigners against sexual misconduct at universities want to prevent NDAs being used to hide criminal conduct and institutional mishandling of harassment and discrimination complaints. They are calling for a code of conduct governing gagging clauses and want the government to enforce this with legislation. An inquiry by the women and equalities select committee may add to the pressure on ministers and higher education regulators to take robust action.

But NDAs are just one way in which universities manage to silence victims and suppress scandals. Many victims have told me how they were isolated and gaslighted by the managers they reported complaints to, or were pressured to make informal, rather than formal, complaints by investigators or HR. (Conveniently, many universities do not record informal complaints.) Chapman’s own experience at University College London illustrates the chilling effect of secrecy in university complaints procedures. After she submitted a subject access request for all the documents about her case, the files the university disclosed were heavily redacted. Although she refused to sign an NDA and agreed a confidentiality waiver with UCL, many questions remained about the internal investigation. Only when I queried its findings with UCL did it emerge that the initial findings against her alleged harasser had been overturned in a secret appeal hearing that she had not been party to.

Such clandestine university procedures make trying to uncover the scale of harassment and bullying incredibly challenging and frustrating. But this problem goes beyond the challenges of investigative reporting: universities are denying victims fair access to information that enables them to properly fight bullies and harassers, which could prevent perpetrators from continuing their abusive behaviour. This has to stop.

• David Batty is a Guardian news editor and writer