The police or security services supplied information to a blacklist funded by the country's major construction firms that has kept thousands of people out of work over the past three decades.

The Information Commissioner's Office (ICO) has revealed that records that could only have come from the police or MI5 have been discovered in a vast database of files held on 3,200 victims who were deemed leftwing or troublesome.

The files were collected by the Consulting Association, a clandestine organisation funded by major names in the construction industry.

Its database was seized nearly three years ago, but the extraordinary nature of the information held has only now emerged, following an employment tribunal for one of the victims, Dave Smith, a 46-year-old engineer who had a 36-page file against his name and was victimised repeatedly for highlighting safety hazards on sites, including the presence of asbestos.

David Clancy, investigations manager at the ICO, told the central London tribunal adjudicating on Smith's claims against construction giant Carillion that "there is information on the Consulting Association files that I believe could only be supplied by the police or the security services".

Speaking to the Observer, Clancy added: "The information was so specific and it contained in effect operational information that wouldn't have formed anything other than a police record."

The scandal will be thrown open to further public exposure in the coming months as a class action by 100 victims against at least 39 companies is set to be pursued in the high court by Hugh Tomlinson QC, currently counsel for several of the phone-hacking claimants. The revelations will inevitably raise fresh questions about the probity of the police in a week in which its relationship with major news corporations, and News International in particular, has come under sharp focus. Last week the Leveson inquiry heard that the police were investigating a "network of corrupt officials" as part of their inquiries into phone hacking and police corruption.

Clancy said he was unable further to trace the specific sources of the information held in the Consulting Association files because it touched on individuals living across the country and stretched over three decades.

He added that the relationship between the Consulting Association and the police and security services appeared to have been nurtured when the organisation went under an earlier guise as the Economic League, at a time when the state was keen to liaise with major building firms to discover as much as it could about Irish construction workers amid the threat of IRA terrorism.

John McDonnell MP, who first raised the issue of blacklisting a decade ago, said he would demand a debate in parliament on what he described as collusion. "I am outraged at the systematic abuse of people's rights. This has destroyed people's lives, broken up families, ensured that people have not been able to earn a living. It has devastated people year after year, and nobody has listened to us. No one has been willing to believe the extent to which there has been collusion between police, security services and companies. It is all about the ability of companies to exploit workers and destroy anybody who stands up against them."

The existence of the secret blacklist was first exposed in 2009, when ICO investigators from the Information Commissioner's Office raided an unassuming office in Droitwich, Worcestershire.

The investigators uncovered an extensive database that was used by construction firms to vet workers they deemed to be trade unionists and troublesome – usually trade unionists. More than 40 construction firms, including Balfour Beatty and Sir Robert McAlpine, had been funding the confidential database, which recorded workers' trade union activities and conduct at work.

The Consulting Association was closed down and a 66-year-old private investigator, Ian Kerr, was fined £5,000 for administering the database, although the construction firms escaped prosecution. At Smith's tribunal, Carillion admitted that two of its subsidiaries covertly supplied information to the database to "penalise" Smith for being a trade unionist, even though he had "reasonably brought health and safety concerns to their attention". However, Smith lost his claim for £175,000 in lost income because he worked through an agency and was not directly employed by Carillion.

A spokesman for the Independent Police Complaints Commission said it had not yet received a complaint from the victims or the ICO.