David Bernstein has attacked comments from the Society of Black Lawyers over its record on racism as "ill-informed and unhelpful". The Football Association's chairman was responding to remarks made by the pressure group's chair, Peter Herbert, following a police decision to drop their investigation into allegations against the referee Mark Clattenburg.

Although Chelsea made a complaint to the FA in October alleging the referee had used inappropriate language towards Mikel John Obi, they did not contact the police. The SBL did make a complaint to the police, based on media reports of the alleged incident.

Scotland Yard announced on Tuesday that no action would be taken "because no victims had come forward".

Herbert said on Wednesday: "It would appear that there is a cosy little agreement between Chelsea FC and the FA not to report these matters to the Metropolitan police but to have them dealt with solely by the FA.

"The FA have a dreadful record of indifference on hate crime generally; failing to challenge anti-Semitism at Tottenham Hotspur and at other grounds; eventually finding John Terry made a racist remark but remarkably found him not to be a "racist"; whilst the derisory penalty of a four- or eight-match ban [Luis Suárez] is believed to be a suitable punishment for what in any other industry would be summary dismissal for gross misconduct."

Herbert also claimed on Sky Sports News that the FA were "institutionally racist … They don't even implement what the Stephen Lawrence inquiry [MacPherson Report] recommendations were about how to report a racial incident, whether the victim or any other person believes it is.

"You will not find that on the FA website or any of the training given to referees, managers, coaches or players. It isn't there. It should be.

"If an institution fails to follow 14 years after the MacPherson Report's recommendations and it has a history of under-reporting race hate, what would you call it? We would call it institutionally racist."

Reacting to the statements, Bernstein said: "These ill-informed and unhelpful remarks are at odds with the progressive and responsible approach being followed by the game with the support of government to deal with these serious issues.

"The FA will continue working to strengthen processes to eradicate all forms of discrimination in football."

Clarke Carlisle, chairman of the Professional Footballers' Association, defended Chelsea and the FA. "I think it would have been better if it remained within the remit of Chelsea and the FA to put any subsequent case to the Met," he said.

"If we reported all incidents from third-party evidence, there would be many investigations the police would have to go through. We have to have faith Chelsea have reported the incidents in good faith and that the FA will deal with it accordingly, and report it to the police if necessary."

The FA is expected to announce the outcome of its investigation into Chelsea's complaint within 48 hours.