Police deployed 24 undercover officers to infiltrate a small leftwing political party over a 37-year period, the Guardian can reveal.

The police spies infiltrated the Socialist Workers party (SWP) almost continuously between 1970 and 2007, often with more than one undercover officer embedded within the party.

Four of them deceived women into sexual relationships while using their fake identities. One spy met one of his wives during his deployment and had a child with her.

About one-third of the total number of undercover officers embedded in political organisations that have so far been publicly identified infiltrated the SWP, a Trotskyite party of a few thousand members that advocates the abolition of capitalism through revolutionary means.

The scale of the infiltration of the SWP – far larger than any other political organisation – is revealed in a database compiled following investigations by the Guardian and the Undercover Research Group, a network of activists that scrutinises police espionage.

The database lists 124 groups that have been spied on by undercover police officers since 1968.

The next-biggest infiltration of a specific organisation was against the campaign opposing the Vietnam war, which was penetrated by nine undercover officers between 1968 and 1972.

The database is incomplete as the full list of groups that were spied on has yet to be established.

The list so far compiled, however, suggests police spies overwhelmingly monitored leftwing and progressive groups that challenged the status quo, with only three far-right groups infiltrated – the British National party, Combat 18 and the United British Alliance.

Undercover officers spied on 22 leftwing groups, 10 environmental groups, nine anti-racist campaigns and nine anarchist groups, according to the database.

They also spied on campaigns against apartheid, the arms trade, nuclear weapons and the monarchy, as well as trade unions. Among those spied on were 16 campaigns run by families or their supporters seeking justice over alleged police misconduct.

According to the database, police spied on 12 animal rights groups and eight organisations related to the Irish conflict.

The database also draws on disclosures made by a public inquiry led by a retired judge, Sir John Mitting, which is examining the covert infiltration of political groups over the past 50 years.

The much-delayed inquiry is still in its preliminary stages and is due to start hearing evidence in public next year.

Mitting and his team have been given the task of scrutinising a range of misconduct, including the frequent deception of women into intimate relationships.

At least 144 undercover officers are estimated to have been deployed to infiltrate political groups since 1968, and 65 of them have been unmasked so far.

Mitting’s inquiry has disclosed that the police spies collected and stored information about the activities of more than 1,000 political groups in that period, although it has not published a list of the groups.

The undercover officers infiltrated specific groups, but also gathered information on a range of other organisations. The police spies adopted fake identities to infiltrate political groups, usually for periods of up to five years, reporting back to their supervisors on what the activists were doing and what protests they were planning.

The extent of the infiltration of the SWP leaves the police facing questions about why they believed such large a number of deployments was necessary.

The SWP’s members have often been conspicuous on demonstrations with their red-and-white placards. However, an assessment by Scotland Yard submitted to the public inquiry concluded the party “does not have a recorded propensity for violence, however, some of its members will have convictions at an individual level”.

Other leftwing groups such as the Socialist party (formerly Militant) were infiltrated by undercover police, but it is unclear why the SWP appears to have been singled out for such intense monitoring.

The first police spy sent to infiltrate the SWP started his deployment in 1970, using the alias of Stewart Goodman. The group was then known as the International Socialists before changing its name to the SWP in 1977.

Another spy – deployed between 1974 and 1977 – was withdrawn by his managers after he told a colleague he had fallen in love with a female activist and wanted to disclose to her that he was an undercover officer.

According to a brief account published by the inquiry, he met his third wife while undercover. They had a child, although it is not clear if he had left the police by that time.

Another spy – who used the fake identity of Vince Miller – has admitted he had two sexual relationships with activists while he infiltrated the SWP between 1976 and 1979.

Two police spies using the fake names of Alan Bond and Michael Hartley in the 1980s also deceived women into sexual relationships. Like many of the police spies, they stole the identities of dead children to develop their fake personas.

Charlie Kimber, the SWP’s joint national secretary, said the level of infiltration revealed the state’s fear of the left. “The whole way the police operated was disgraceful. In many ways, the most appalling aspect of it were the women who were tricked into what they thought were real relationships,” he said.

The Metropolitan police said the work of its two now-disbanded undercover units, the Special Demonstration Squad and National Public Order Intelligence Unit, will be fully scrutinised by the public inquiry. It said it “is providing every assistance so the inquiry can fully address the key issues it has identified, and can fulfil its terms of reference”.