For four years, Officer A lived a secret life among anti-racist activists as they fought brutal battles with the police and the BNP. Here he tells of the terrifying life he led, the psychological burden it placed on him - and his growing fears that the work of his unit could threaten legitimate protest

They got drunk together, stood shoulder to shoulder as they fought the police and far-right activists, and became so intimately acquainted with each other's lives that in the end they were closer than brothers. But it was all a sham. Hidden among the close-knit and highly motivated group of violent far-left activists was a serving police officer, operating deep undercover, whose presence was intended to bring the group to its knees.

That man, known only as Officer A, has now come forward to give his account of the years he spent working for Scotland Yard's most secret unit, the Special Demonstration Squad (SDS), on a mission to prevent disorder on the streets of London. For four years in the mid-1990s, he lived a double life six days a week, spending just one day a week with his wife and family.

Week after week, year in and year out, he lived and breathed the life of a hardcore Trotskyist agitator with a passion for heavy drinking, a deep-seated hatred of the police and a predilection for extreme violence. It was a persona that took him to the heart of some of the most violent groups in the capital at a time when tensions between extreme left and extreme right were at their peak.

"I never had any respite when I was back at home. I simply couldn't relax," said Officer A. "The respite for me was being back in my undercover flat because that was where I was supposed to be. Even if my targets were not there, I felt more at ease. I had a really good time with my targets and enjoyed their company enormously – there was a genuine bond. But I was never under any illusion about what I was there to do. They were not truly my friends. The friendship would last only up until the point when they found out what I really was. I was under no illusion about what would happen to me if they did."

The SDS started life in 1968 after anti-Vietnam war protests turned violent in Grosvenor Square, London, outside the US embassy. No one had been prepared for the unprecedented level of violence directed at the police. It was a wake-up call to senior officers in the Metropolitan police who realised they needed a new way to gather intelligence about the hate-filled "subversives" they now had to deal with.

Known as the "hairies" because they had to look like the agitators they were mixing with, the 10 undercover officers who made up the unit were given new identities, flats, vehicles and "cover" jobs while working in the field for years at a time. Officer A had joined the Met in 1986 straight from school. Having discovered an interest in political ideology and public affairs, he initially wanted to join the security service but found that entry was barred to all but Oxbridge graduates.

His best chance of doing that kind of work, he was told, would be to join the Met and apply for a job in Special Branch, essentially a wing of MI5. Having distinguished himself during cadet training and his two-year probation, he joined Special Branch after four years in uniform and spent three years working to counter Irish terrorism before being recruited to the SDS.

"The day you join the SDS, you have a big leaving do. Although you're still a police officer, being in the SDS means you won't be seeing any of your police friends for at least five years. During your deployment, your only official link to the Met is your payslip," he said. SDS officers use the same methods as Frederick Forsyth detailed in The Day of the Jackal to choose their new identities. This involves applying for the birth certificate of someone who died at an early age and using this to fabricate a cover story. "That first step gives you a name, a flat, a vehicle and a life story that makes it all real."

Before being deployed in the field, SDS officers are taken into a room and interrogated about every aspect of their cover story by two experienced operators. If they pass this test, they are then told the cautionary tale of how an SDS officer's cover was blown when his targets asked him to explain the death certificate for the cover name he was using and had to jump from a second-floor window to escape.

Officer A's initial target was a young student union activist who was a key member of an up-and-coming Trotskyist organisation that had led a violent protest against BNP paper-sellers in Brick Lane, east London. The organisation was considered one of the most violent in the capital at the time and its leader soon became a priority SDS target.

Officers normally spend months getting to know their targets and winning their confidence. In the case of Officer A, however, it took far less time. "I enrolled at the college where the target studied and on my first day there I heard a bit of an altercation while I was in the queue for lunch. A few things were said to one of the female staff, some of which had a bit of a nasty racial overtone. I pushed forward to intervene. It all got a bit heated and the guy who was giving the abuse took a swing at me. Big mistake. He was soon on the floor, out cold.

"A close friend of my target was in the canteen at the time and had seen the whole thing. A couple of hours later, I was in the student union picking up some passes and the bloke who had watched me fight was there again. He turned to my target, who was sitting next to him, and said: 'That's the one I was telling you about.' A little later, my target came over and introduced himself."

Officer A eventually agreed to attend a small demonstration the following weekend. When the event turned violent, he found himself standing next to his target and others from the group as they launched a series of attacks on uniformed police. "The one thing all these groups have in common, both on the left and the right, is a total hatred of the police," he said. "It means you are having to maintain a false identity in a completely hostile environment. Under those circumstances you have no choice but to engage in acts of violence. That day developed into a major ruck. At the end no one would have believed I was a police officer.

"If anyone had accused me [of being a police officer] there would have been a dozen people willing to come forward and swear it wasn't true. We were all buzzing when it was over," said Officer A. "We couldn't wait for the next event. Because of what we'd all been through, I was accepted by them right away."

Officer A wasn't the only one attacking his former colleagues. At the time of his deployment, other SDS officers had infiltrated opposing right-wing groups such as the BNP and Combat 18, as well as other far-left groups. It was a time of extreme racial tension and violent clashes with the police and rival political parties were rife. Two weeks later, Officer A took part in a much larger, far more violent, protest in Welling, south-east London, against a BNP-run bookshop that served as the party's headquarters. Intelligence he obtained revealed that the demo was to be far larger than had been expected and that a particularly violent faction was planning to storm the bookshop and set fire to it, trapping any BNP members inside.

As a result, police leave was cancelled for that weekend and more than 7,000 officers, including a large mounted contingent, were deployed. Instead of being spread out along the entire route, police focused on blocking the main roads leading to the bookshop and forcing the march along a route that would take it away from its target. A violent confrontation ensued with a group of hardcore protesters – Officer A among them – attacking the police lines in an attempt to break through. Dozens of police and protesters were injured in the clashes.

Despite the violence, the operation was deemed to be a success for the police and Sir Paul Condon, the then Met commissioner, went to meet members of the SDS to thank them.

"I didn't have any qualms about what I was doing," said Officer A. "I was clear that my role was to target subversives and prevent disorder. The consequences of that day would have been far worse had the SDS not been involved."

At that time, some of the SDS officers were known as "shallow paddlers" because they spent only limited time with their targets. Others, like Officer A, were "deep swimmers" who immersed themselves in the role. During one operation to infiltrate an Animal Liberation Front cell, one officer is said to have lived in a squat for 18 months, virtually 24/7.

As months turned to years, Officer A's personal life was beginning to suffer, and his relationship with his wife and children was under particular pressure. One major cause of stress was that he was spending so much of his time fighting with fellow police officers and was now on the wrong side of a riot shield. "It was a total headbender," he said.

Once inside the groups they were ordered to infiltrate, it was relatively easy for SDS officers to rise to the top because they were often prepared to work long hours on boring, administrative jobs. Often they tried to become membership secretaries or treasurers, where their position gave them access to the records and secret agendas that were so useful to the security services. Often more efficient than those around them, operatives had to strike a balance to ensure they did not end up running the organisations they were trying to destroy.

Protesters clash during an anti-racist march in Welling, south-east London, in 1993. Photograph: STEFAN ROUSSEAU/PA NEWS

In the aftermath of the Welling riot, senior Met officials began to express concern that the so-called "black campaigns" that had sprung up in the aftermath of the murder of Stephen Lawrence in April 1993 had the potential to lead to further bloodshed on the streets of the capital. "It had only been a couple of years since the beating of Rodney King led to the riots in LA. When young black men started dying in police custody and racist murders were going unsolved, a lot of people were getting increasingly angry," said Officer A. Fearing they were in danger of losing control, the SDS decided to target these new groups.

Having won the trust of several high-profile anti-facism and anti-racism activists on the far left, Officer A was ideally placed. Over the next two years he worked his way up to become branch secretary of Youth Against Racism in Europe, a leading anti-racist organisation that was a front for the far-left group Militant. Getting alongside these new targets called for a different approach, said Officer A. "You get given a file on your target that tells you everything you need to know. You become that person's brother. You know everything that makes them tick. You know how much they like to drink, you know where they like to drink. You know what kind of music they like, you know what kind of women they like. You become the brother they never knew they had. None of it is ever said to the target, it's far more subtle than that. The first time they get in the car, it will be just the right kind of music playing. The first time a redhead walks by it will be: 'God, I'm really into redheads.' It's all done fantastically cleverly.

"When your target is a man, it is just a matter of becoming his best friend. If your target is a woman, that becomes impossible. SDS officers would get together for regular meetings and you always knew if something was going on. If someone started talking about getting good information from a female target, we all knew there was only one way that could have happened. They had been sleeping with them." He himself had slept with two members of his target group. Although not officially sanctioned, such activity among SDS officers – both male and female – was tacitly accepted and in many cases was vital in maintaining an undercover role. "You can't be in that world full-time for five years and never have a girlfriend or boyfriend. People would start to ask questions," said Officer A.

But the pressures continued to grow. "At first, I could convince myself that my job was about fighting subversion, but once I began targeting the groups set up to win justice for those who had died in police custody or had been victims of racism, it was clear that what the loved ones of the deceased wanted was justice. My presence in the groups made that justice harder to obtain. The remit was to prevent disorder, but by providing intelligence you rob these groups of the element of surprise. If every time they have a demonstration the agitators are prevented from causing trouble, they are less effective. Once the SDS get into an organisation, it is effectively finished.

"If I were a regular police officer and I wanted to plant a bug in your house or your office, I would need to get all kinds of permissions. But the SDS can put a person in your car, in your house, in your life for 24 hours a day for five years and nobody outside the SDS will know anything about it.

"While I was undercover, my targets would refuse to talk on the phone because they felt it wasn't safe, but they had no qualms about inviting me into their homes to talk about their plans. I couldn't get over the irony of it. If the SDS had been in existence at the time of the Suffragettes, their campaigns would never have got off the ground and they would have been quickly forgotten."

The constant strain of living a double life was also beginning to take its toll. "I couldn't get out of role. Even after 18 months I was having trouble leaving the undercover persona behind. One time I was out swimming. Someone said something derogatory and my angry persona took over. It was an immediate reaction. There was blood everywhere."

Before they were deployed, every SDS officer was visited at home to ensure they were married. "They introduced that rule after one officer refused to come out of the field. It turned out he just enjoyed being with his contacts so much that he was willing to give up his police salary and everything that went along with it in order to stay with them. Now you have to be married on the basis that, if you have something in the real world to come back to, you are less likely to want to remain in role. That's the theory."

The pressure began to become intolerable when a public inquiry into the murder of Stephen Lawrence was announced. "There was concern that my role within the campaigns might emerge during the inquiry," he said. "In the end the SDS decided not to disclose it themselves. Because the remit was to prevent disorder, it was felt that if it had emerged at the time, it would have led to more violence.

"Looking back, I should have done something. I should have dealt with this 11 years ago. I am coming forward to get closure for the things I did back then. By the end I'd spent four years fighting the police. When I came back to Special Branch I had to suppress who I was. I was no longer the same person. I hated the job and everything about it."

Officer A was later diagnosed as suffering from post-traumatic stress disorder. He sued the Met and received an out-of-court settlement. The Metropolitan police, meanwhile, has refused to comment on any matters connected to the SDS.