(CNN) The manager of an English football team that walked off the pitch following allegations of racist abuse from supporters says those fans found guilty should be "banned for life."

Two men have been arrested by police after investigations into racist comments made during an English FA Cup match between Haringey Borough FC and Yeovil Town on Saturday.

The men -- aged 23 and 26 -- were arrested on suspicion of racially aggravated common assault and are both in custody at a police station in Somerset.

Metropolitan Police officers are leading the investigation with the assistance of Avon and Somerset police.

"There's no room in society for people like this," Haringey manager Tom Loizou told CNN Sport's Kate Riley. "The way they made a few of our players feel, there's no room for these type of people in any kind of sport, not just football."

Read More