Birmingham, England (CNN) Khalid Masood, the man responsible for the worst terrorist attack in London in more than a decade, kept to himself in recent years, neighbors say. But he had a checkered past involving a series of aliases and a string of convictions for violence and possession of weapons.

Police kept his identity quiet at first as they sought to track down his associates and search related properties.

But a fuller picture is emerging of the 52-year-old who drove an SUV onto a sidewalk Wednesday on Westminster Bridge, killing four people and injuring dozens more before being shot dead by a firearms officer just inside the grounds of the Houses of Parliament.

The early years

Masood was born on December 25, 1964, in Kent in southeast England. His birth name was Adrian Russell Ajao, although he went on to use multiple aliases.

As a boy, Adrian Ajao -- as he was then known -- was a popular pupil who "liked to party" and was a keen soccer player, former high school classmate Stuart Knight told Britain's Press Association news agency.

The pair attended Huntleys Secondary School for Boys in the well-heeled town of Tunbridge Wells, Kent, for five years, Knight said. They left the school -- which has since closed -- in 1981. An undated school photo shows Ajao, smiling and with his arms folded, with members of a soccer team.

Khalid Masood, then known as Adrian Ajao, appears in a school photo with the soccer team.

"He was a very nice guy, down to earth, liked by everyone around him," Knight told PA. "There was nothing unassuming about him, he was a very good sportsman, his mother was a Christian, he was an all-round nice guy."

Criminal convictions

It's not clear how Masood developed from being a well-liked, athletic schoolboy to a man with a string of criminal convictions.

JUST WATCHED London Attack: An animated map Replay More Videos ... MUST WATCH London Attack: An animated map 00:55

His first conviction -- for criminal damage -- was in November 1983 when he would have been 19, according to police. It was followed by convictions for assaults including causing grievous bodily harm, possession of offensive weapons and public order offenses.

His most recent conviction was in December 2003 for possession of a knife, police said.

However, he had not been accused of any terrorism offenses.

Police first identified him as Khalid Masood but said he had also gone by the name Adrian Elms.

report published in November 2000 in the East Sussex newspaper The Argus said Adrian Elms, then 35, was given a two-year prison term for slashing the face of a cafe owner in an argument in a pub.

Masood had prior convictions for violent offenses but none for terrorism.

According to the report, Elms, from the village of Northiam, near Rye, admitted charges of unlawful wounding and criminal damage at Hove Crown Court. He was one of only two black men living in the village at the time, and there were "racial overtones" to the row, the court heard.

"His wife and family have now become (ostracized) in the village. It is a very small community and his wife and family have been extremely affected by this," his attorney, Alexander Taylor-Camara, told the court.

"He will effectively have to move his family from the village and start to live his life all over again. It will leave the village with a view of black people in the area. The trust that may have been there in the beginning is now completely lost."

Travel to Saudi Arabia

The Saudi Embassy in London confirmed that the London attacker lived in the kingdom between November 2005 and November 2006 as well as April 2008 and April 2009. During that time, Masood was an English teacher, according to a statement published on the state-run Saudi Press Agency.

The embassy statement added that in 2015, he procured an Umra visa -- required for pilgrimage to Mecca throughout the year outside of Hajj -- and visited the country between March 3 and 8.

"During his time in Saudi Arabia, Khalid Masood did not appear on the security services' radar and does not have a criminal record in the Kingdom of Saudi Arabia," it said.

Birmingham connections

After prison, Masood appears to have moved around over the years -- most recently in Birmingham in central England.

Police searched two properties where Masood and his family were thought to have lived -- the Winson Green and Hagley Road areas of Birmingham. Eight of the 11 arrests made as of Friday afternoon in connection with the investigation were in Birmingham.

Police carry boxes this week from a Birmingham address in connection with the London attack.

Police were seen Friday carrying boxes from a modern, redbrick townhouse in a quiet cul-de-sac in Winson Green. Masood is believed to have lived there until December.

Marjoli Gajecka, 26, told CNN she had seen Masood and his family often when she visited her mother, who lives on the same street. Masood was calm and kept to himself, she said, adding: "He was never really talkative with neighbors."

She said Masood had two children, a boy and a girl, whose ages she estimated to be 6 or 7.

A police van is outside a residential building Thursday in Birmingham.

She said she was shocked by the news he had carried out the attack in London. "They just seemed a normal family, a normal Muslim family. We never suspect anything," she said.

Masood wore "Islamic clothes" most of the time, Gajecka said. A woman she presumed to be Masood's wife wore a veil with her face uncovered. She also saw the young girl wearing a headscarf.

Masood and his family would not have stood out in this multicultural, diverse neighborhood. People are close here, popping in and out of each other's homes. Some voiced shock over how the atrocity in Westminster had been brought to their doorsteps.

Police stand guard Thursday at the entrance to an apartment on Hagley Road in Birmingham.

About two miles south, on Hagley Road, is the apartment where Masood and his family are thought to have moved at the end of the year.

Several arrests were made at the property after police raided it Thursday.

Raviyar Sedighi, who owns a shop below the apartment, told CNN he recognized Masood and that "he was a normal customer, normal man."

He knew him as a "bodybuilding man," Sedighi said, going to the gym and buying energy drinks. Sometimes Masood wore a cap, at other times he didn't wear Islamic dress, he said.

Radicalization

The wide-ranging police inquiry is focused on how Masood became radicalized.

Asked about his possible conversion to Islam and subsequent radicalization, Britain's top counterterror officer, Mark Rowley, said: "Clearly that's the main line of our investigation -- is what led him to be radicalized. Was it through influences in a community, influences from overseas or through online propaganda?"

Prime Minister Theresa May said the attacker was known to authorities for links to "violent extremism."

He was investigated "some years ago" by security services but was regarded as a "peripheral figure," May told the House of Commons. He was not part of the "current intelligence picture," and authorities did not know he was about to mount an assault.

A UK official told CNN that Masood's age was not typical for an ideologically driven attacker: "Fifty-two is interesting. Not the usual young pup profile."

The ISIS-linked Amaq news agency asserted the attack was carried out by one of its "soldiers." But that does not necessarily mean the group had any direct connections to Masood, whom it did not name.