Darren Carroll is a slight man with bright blue eyes and an urgent need for redemption. A painter-decorator in his fifties, he has lived in Luton his whole life. He was one of the original members of the English Defence League (EDL), the far-right street movement founded by Carroll’s nephew Tommy Robinson.

Recently, things haven’t been easy. Four months before our meeting at a café near Luton Airport Parkway Station, Carroll had a minor stroke that affected his speech and vision. It was the delayed fallout from an attack in a pub across the road, his local. A stranger, who seemed to know a lot about him, started a conversation. “He showed me his arm. It was tattooed. There was a little bit of white skin left on the whole sleeve,” says Carroll. “He said, ‘Look at that.’ I said, ‘What?’ He said, ‘White is right.’ I said, ‘Nah, mate, I know exactly where you’re coming from. There’s nothing wrong with being white but there’s nothing right with it.’”

The man pretended to leave the pub, then walked back in and hit Carroll hard on the back of the head with his forearm. Afterwards, Carroll suffered persistent headaches. It caused a blood clot that set off the stroke. When we met, he had mostly recovered but was still unable to work.

It was not the first attack. Carroll has also had his front door kicked in. He and his children have received death threats. “This is since speaking up,” he says. “Not leaving – that’s different.”

Carroll looks uncomfortable when we discuss the early days of the EDL. “It was an organic thing,” he says. “Lots of people were involved at the very beginning for different reasons. Personally, I was not happy with the way the town was being run on a political level. Looking back, I was disenfranchised from mainstream politics.”

Luton has the dubious distinction of being a centre of both far-right and Islamist extremism. The EDL began here in 2009, in response to a demonstration organised by Anjem Choudary’s now banned extremist group al-Muhajiroun, which in turn was a reaction against an army regiment marching in Luton.

A counterprotest led to arrests and the EDL was born, with sometimes violent neo-fascist street protests spreading across the country. Robinson insisted from the outset that the EDL was not racist, but only “against the rise of radical Islam”. Carroll says it was local difficulties, rather than national issues such as immigration, that unsettled and motivated him – and he didn’t articulate the core problem as racism against white people, not even to himself. The EDL has never had a formal membership, but the think tank Demos estimated that there were between 25,000 and 35,000 active members in 2011, a loose coalition of football hooligans and far-right activists. Today, the numbers are much reduced.

Carroll’s family was closely involved and it was a while before he realised that the EDL was an extremist, racist group. He describes being at a demo in Birmingham soon after the first protest. “I looked at the other lads there and I didn’t like them. They didn’t smell right for me, as far as integrity goes. I thought, ‘I don’t want this.’” Carroll’s parents are Irish and he considers himself the child of immigrants.

It took several months for him to extricate himself from the group and stop attending demonstrations. “It’s a relationship breaker, so you’ve got to accept that things are broken for ever.” On building sites, he was known as the EDL guy. Work dried up.

Amid attempts to coerce him back into the movement, and concerned about damaging his family relationships, Carroll stayed silent for another year and a half, only starting to speak up a few years after he left the EDL. This triggered a new wave of threats. He reeled off a list of incidents: slashed tyres, smashed windows. “Last week, I got one on Facebook [saying] that I’m a ginger Muslim and I’m gonna get shot. That was someone I know privately, which I don’t take as a threat. Their particular problem seems to be that I’m on record saying I’d have a cup of tea in a mosque and sit down and talk to people.”

Carroll did so after seeing a Facebook post by a local activist, Dawood Masood. Masood had shared a video of an imam in Leicester speaking about terrorist violence, with a message saying that any EDL members were welcome to get in touch. Carroll met him and others from the Muslim community and they discussed ways to make Luton better. He told them that he wasn’t interested in religion, but invited them to what he considers his church: Luton Town FC.

“I had the idea it’s about setting precedents, because you never know who or what that affects,” he says. “I just thought, if I’m seen going to the football with them, it’s going to break a big piece of ice.”

As the EDL evolved largely from a football subculture, this was a bold step. They went to the match. “He’s Luton born and bred and he certainly don’t need his hand held. But I made him as comfortable as possible. Luton scored and he’s jumping up and down, loving it. At that point, I thought: ‘This is really Luton harmony. He’s cheering for the same thing and I’m cheering for the same thing. We’re both happy together at this moment in time. We’ve got things in common.’”

They have been to many matches since, Masood bringing his kids, Carroll his grandkids. Carroll has had a few threatening calls but remains undeterred. “The working-class Muslim lads are working-class Muslim lads. They’ve got all the same problems and social issues as us white, working-class people. It’s not just me or us. It’s everyone.”