Westminster terrorist Khalid Masood was an “apolitical” man who showed no interest in radical Islam in the two years he lived in Luton, his former boss said.

Farasat Latif, a director at language school Elas UK where Masood worked between 2010 and summer 2012, said he knew Masood as a charming, friendly and professional employee who was open about getting his life back on track after a violent past.

In the aftermath of last week’s attack, Theresa May told MPs the security services had been aware of Masood as a “peripheral” figure.

Latif, a trustee of Luton Islamic Centre, said that during the time Masood worked with him, Latif was involved in confrontations with members of the radical Islamist group al Muhajiroun, on occasion driving them away from the street stalls they set up in Bury Park.

“He knew all of this and it went completely over his head, it didn’t interest him,” Latif said. “He must have come into contact with them, because they don’t stand there ... they will go up to you and they will be in your face, you can’t avoid them,” he said. “I remember he did once ask me about them, he said who are they.”

But he added: “Khalid was a middle-aged, middle-class, intelligent black man and these were young, highly unintelligent young Asians. There was no common ground between them. He was apolitical, they were politicised.”

Masood appeared ignorant about many of the issues that extremist groups play on to recruit followers, such as Israel and Palestine or global jihad. “He did not fit to me as a potential extremist in any way, shape or form,” Latif said.

Latif recalled seeing Masood angry only once, when the English Defence League announced a march on Luton. “He was absolutely livid they were allowed to march,” Latif told the Guardian. “‘He said, ‘If they were to come to my town I’d kill them.’ That’s the only time I ever heard him say something aggressive.

“I felt there was a race issue, not a religious issue.”

During the two years Masood worked at the school, he would drop into Latif’s office a couple of times a week for a chat. “He basically told me he had a very troubled life, he’d been involved with a lot of violent crime, he was a very violent person,” Latif said.

After one stint in jail, Masood told Latif he “made a firm resolve not to get back to crime and clean up his life and become a better person”.

Latif added: “I don’t know if he was a Muslim at that point, but he would definitely have become a Muslim shortly afterwards.

“He came across as a very mature, middle-class man who was very focused on his family and his career and also being a good Muslim.” After two years he left for Birmingham.

When news of Masood’s name was published last Thursday, it did not occur to Latif it was his former employee. “I thought, oh no, I’d better ring Khalid and tell him that killer in London’s got the same name as you ... It’s a fairly common name but I knew it was a Khalid Masood, I knew he was black, but I still thought it can’t be the same guy, what a coincidence.”

The confirmation it was the former employee who had killed four people and been shot left him and others involved with the school “in a collective state of shock … I was distressed, I was angry, I was sickened.”