David James, one of England's most decorated black footballers, last night accused anti-racism groups of having "an agenda to keep themselves in existence".

The goalkeeper, capped 53 times by England and still playing at the age of 42 for Bournemouth in League One, added: "I think the organisations which have done so much good [combating racism] on the terraces are still employed looking for stuff to be shouted about. I think that some people have an agenda to keep themselves in existence. I struggle with the racist issue in football because as a player I don't see it. That's not because I've got my head in the sand. In the earlier days, yes, but the game's changed. It is not what it was. I don't believe it is any more racist than society is."

James did not name any organisation, but was almost certainly talking about Kick It Out, the anti-racism organisation dedicated to eradicating racism in football. He added: "There are some wonderful organisations out there which have helped football become a much more enjoyable game for everyone, from stuff in the crowd being aimed at players – that's gone, or pretty much gone. I don't hear it any more. I don't look at myself as any different from the guy who gets changed next to me, and I'm not going to fly anyone's flag in order to join some 'gang', which doesn't need to be joined."

James stressed it was right for John Terry to be punished over the racist language he directed at Anton Ferdinand, but felt the Football Association should have acted without waiting for the police investigation to be completed.

"The JT thing, it should be dealt with – if someone said that in the stand they would be ejected – but it could've been dealt with quietly and done rather than it being a six- or eight-month thing. It was allowed to fester. The governing body should have said, 'An incident has taken place, racism occurred on a football pitch.' They should have dealt with it there and then. Whether he is or isn't racist, he used language on a football field which should not have been used and that should have been dealt with."

James, who wants to become a manager and is doing his coaching courses, also said he did not believe the shortage of black managers was down to racism. "I think it is whether you are good enough. Look at Chris Hughton, he's managed three clubs and been decent, there have been others who weren't good enough."

James echoed the words in The Independent last week of Keith Curle, the manager of Notts County and one of only four black managers in the professional game. "I have not seen many ethnic players on the courses I have been on in the last two years. If you want to become a manager, then give yourself a chance. If you don't want to go on the courses, then moan about not getting jobs, well, probably the reason you haven't got a job is you haven't been on the courses."