One October evening in 2017, the counter-extremism expert Julia Ebner put on a wig and a pair of glasses, walked into a pub in Mayfair, London, and sat down with 20 strangers. She was attending the inaugural strategy meeting for the UK members of Generation Identity, a pan-European far-right network known for spreading anti-immigration ideas and stoking fears about a “great replacement” in which white populations become minorities. Ebner had spent months infiltrating the group’s encrypted, invite-only messaging channels under an alias and striking up a rapport with members; now she was meeting them in person.

I meet Ebner at the same pub, only this time it is mid-morning and, save for the barman polishing furniture, there is no one else here. “This feels weird,” she says. “I was really scared walking in that night so I remember it as having a dark and oppressive atmosphere. I’m surprised at how unremarkable it feels today.”

Her first meeting with Generation Identity was brief: they talked disparagingly about the anti-racism organisation Hope Not Hate; the best way of vetting new recruits; white supremacist literature; and where they were going on their holidays (Hungary was a popular choice). More nerve-racking, says the 28-year-old, was the second meeting the next day in a flat in Brixton, south London, which was led by Martin Sellner, the head of Identitäre Bewegung Österreich, Austria’s Identitarian Movement. Ebner feared her cover would be blown as she and Sellner had both featured in a report on extremism on Newsnight the previous year, but, as they shook hands, he didn’t appear to recognise her. Attendees were asked to fill in a questionnaire about their background and ideology, after which talks were given by Sellner and Brittany Pettibone, Sellner’s girlfriend (and now wife). Among the topics was the danger posed by infiltrators from anti-racist organisations, security services and the media. “I thought: ‘Oh God. Are they going to pull everyone’s hair to see who’s wearing a wig?’” Ebner recalls.

Her experiences are detailed in her new book, Going Dark: The Secret Social Lives of Extremists. Along with recording her meetings with Generation Identity, it tells how, over the course of two years, she inveigled her way into the chatrooms of the Trad Wives movement, including the anti-feminist Red Pill Women, and the Terror Agency Sister group, a women-only group for supporters of Islamic State. In US “alt-right” channels, she eavesdropped on arrangements for the 2017 Unite the Right rally in Charlottesville. Ebner had planned to attend but changed her mind after people began posting pictures of firearms. (The event ended in tragedy after a man deliberately drove a car into a group of protesters, killing a young woman and injuring scores more.)

Creating a false identity is like inventing a character for a book – you have to know the person’s background and desires

Ebner, who speaks five languages, is a research fellow at the Institute for Strategic Dialogue in London, a counter-extremism organisation founded in 2006 with a focus on combating the spread of propaganda, hate speech and disinformation online. Her first book, The Rage (2018), was about the reciprocal relationship between far-right and Muslim extremists and how, she says, “each exploits the fears that the other side provokes”. She was inspired to write it after the murder of the Labour MP Jo Cox in 2016 by a rightwing extremist. Before Cox’s death, her research was broadly centred on jihadism. “But then I shifted focus. I couldn’t stop thinking about what had happened to her, and I realised how little attention was being paid to the threat from the far right.”

Going Dark builds on that research by revealing how extremist groups from across the ideological spectrum harness technology to spread the word, and exposes their methods of radicalisation, from online recruitment and networking to mobilisation and intimidation tactics. “The radicalisation engines that today’s extremists are building are cutting edge: artificially intelligent, emotionally manipulative and socially powerful,” Ebner writes in the introduction.

In the course of her research, Ebner has been known to juggle five online identities simultaneously. To do this successfully, she says, “you have to create a story for your character. It’s a bit like inventing a character for a book. You have to know the person’s background but also her desires. What does she want? What makes her angry? And then it’s about maintaining that presence online for a period of time so people get to know you.”

Newcomers to far-right extremist networks can initially be asked to prove their whiteness by taking a photo of their wrist along with a timestamp and the group logo. Ebner was once asked to submit a genetic test to prove her white ancestry (she used a fake one). Some networks even instruct new recruits to launch an online hate or disinformation campaign against, say, a political campaigner – “just to show that they are willing to go there”, Ebner says.

Is there a thrill in pulling the wool over people’s eyes and potentially putting herself in danger? “I would be lying if I said there wasn’t an element of that; there is an adrenaline rush in doing any kind of undercover work,” Ebner says. “To be able to get inside a movement while being completely opposed to the ideologies, that can feel good. Though there may be better ways to get that kind of kick … such as jumping out of a plane.”

Facebook Twitter Pinterest Neo-Nazis clash with anti-fascist counterprotesters at the Unite the Right rally in Charlottesville, 2017. Photograph: ddp USA/REX/Shutterstock

Nonetheless, there have been times when fears about her personal safety have almost overwhelmed her. In 2017, she wrote a comment piece for this newspaper on the mainstreaming of far-right groups in which she observed that the English Defence League co-founder Tommy Robinson’s supporters included prominent US alt-right leaders. At the time, she was working for the counter-extremist thinktank Quilliam.

The next day, Robinson and a camera crew entered her office to demand that she justify linking him to white supremacists. The incident was uploaded on to the internet as part of Robinson’s Troll Watch series. It was followed by a campaign of online harassment led by others against Ebner, which included threats of sexual violence and death. “That was quite traumatising,” she recalls. When leaving work, she would worry that she was being followed, “or that when I got home there might be someone waiting for me. But in the end I came to the conclusion that that’s exactly what they want. It is their goal in their intimidation campaigns to make people quit their jobs, to make them too afraid to criticise them or expose their manipulative tactics. What kept me going was not wanting to give in.”

Going Dark doesn’t underestimate the continuing threat of jihadism – a fortnight after we meet, Sudesh Amman, who had already been convicted for terror offences, stabbed two members of the public in Streatham, south London, before being shot dead by police. Ebner’s focus, however, is on the methods used by everyone from jihadists to conspiracy theorists to white supremacists and anti-feminists to disseminate, radicalise and recruit. It explains broader concepts used by extremists, including “redpilling” which, inspired by the 1999 film The Matrix, sees recruiters persuading potential sympathisers of an alternative reality in which they are oppressed and threatened by particular groups, and the “gamification” of discourse, in which terrorists adopt or are given heroic names from computer games, or are given scores for injuring or killing. By adopting the language of gamers, the lines between fantasy and reality become dangerously blurred among perpetrators and sympathisers.

Ebner anticipates a near future in which AI tools bolster online extremist campaigns with “deepfakes” – fake newspaper articles and audio and film footage that has been seamlessly doctored. Matters aren’t helped by the slowness of policymakers and tech firms to rid platforms of harmful content, or the fact that extremists manage to stay one step ahead by developing new methods to escape detection.

Does Ebner’s work makes her feel bleak about the future and our ability to combat extremism? “Yes,” she replies. “There are times when I feel pessimistic about how these extremist movements have the potential, in the medium and long term, to completely destabilise our democracies, not just through intimidating opponents but manipulating normal, average users and radicalising sympathisers.” But, she adds, researching the book also made her see a human side to people with extreme viewpoints. However disgusting she found their ideologies, she says she felt “empathy, even sympathy, for people’s fears. Because in the end there was always some kind of fear, or lack of love or recognition, that was driving individuals into those networks.”

Extremist movements could completely destabilise our democracies, intimidating opponents and radicalising sympathisers

On coming across young people in neo-Nazi channels (the youngest she encountered claimed to be 14), Ebner had to resist the urge to engage with them individually and attempt to change their minds: “I had to keep telling myself that I will have a bigger impact by gathering information.” A young person’s professional future can be wrecked by associations with extremist groups, something their leaders often count on to ensure lifelong commitment. “Isis were known for putting their newest faces on their propaganda material,” she says. “It’s very hard to reverse that.”

Given the threats she has endured online, Ebner is understandably cautious about sharing details of her personal life. What she will say is that she was born and raised in Vienna – her mother is an actor turned acting coach and her father a healthcare consultant. After secondary school, she studied for a degree in international business administration and philosophy in Vienna, though, she notes, “I wasn’t really interested in the business part. I just did it as I thought I could earn money.” She subsequently got a scholarship to Peking University for an MA in international relations and history. She completed her final year at the London School of Economics, where she shifted her focus to political Islam and the history of jihadism, and got her first job at Quilliam soon afterwards.

Ebner recalls that, while she was growing up, there was what she describes as a “very utopian view of the future of the internet … There was this sense that it would enable the democratisation of knowledge, make the world more tolerant and build bridges rather than build tensions and split communities.” But by the early 2010s, the shine had begun to come off. “The rate of technological change scares people,” she notes. Its effect on society “is something that’s on the mind of most people of my generation”.

I ask if the months spent trawling message boards with often horrifyingly violent and racist content takes its toll. “It does,” she says, “though it’s a creeping kind of trauma. Research by psychologists suggests that it can have long-term effects that can take a while to appear. I try my best to avoid social media and have online detoxes regularly, though that can be difficult. When working undercover, you need to maintain a presence online. You can’t just disappear.”

Ebner is also keenly aware of “how distorted the whole online space is, and how that doesn’t reflect general opinions. The extremist voices usually get a megaphone for their posts.” As if to underline this, she says that, as a young Austrian woman living and working in London, she has found Britain enormously welcoming. “Even with the divisions caused by Brexit, there’s this real sense that [in the UK] you can be who you are and you can be respected regardless of your nationality, your age, and your gender,” she explains. “But the thing I love best is the dark humour. You don’t get that anywhere else, and I wouldn’t be able to do my work without it.”

• Going Dark is published on 20 February by Bloomsbury (£16.99). To order a copy go to guardianbookshop.com. Free UK p&p on all online orders over £15.