When Tiger Woods was 14 he was acutely aware of the racist undertones that lingered in country clubs across the US. The world No1's success ever since may have challenged golf's outdated perceptions of race, but 23 years after conducting that interview there remains a worrying shortage of non-white amateurs playing the game in England.

"I can always feel it, I can always sense it," said Woods in 1990. "People staring at you and thinking: 'You shouldn't be here.'" It is a sentiment mirrored in figures obtained from Sport England, which reveal that only 2% of a total 850,500 people who play on a weekly basis are non-white. It is an alarming statistic and falls significantly below the equivalent figures in football, rugby union, cricket and tennis.

There are certain things one must put up with on a golf course to appease the elitist status quo. Yet although it is possible to bite your tongue while getting barracked for walking into the clubhouse with a shirt untucked or having the temerity to wear a cap indoors, ignoring the game's stark failure to embrace all demographics across society is harder to stomach.

There are clubs across England that still have no members from ethnic minority backgrounds, while high-profile incidents such as Sergio García's comments on Tuesday night and the assertion from Woods's former caddie Steve Williams in 2011 that he would "like to shove it up that black asshole" have cast shadows over the game.

It was only last year when Augusta National first permitted two women to become members, meanwhile, one being the former US secretary of state Condoleezza Rice, following concerted pressure.

Jaz Athwal became the first Asian captain of a golf club in England 13 years ago and organises the annual UK Asian Open. Based in Bradford, Athwal believes that racial abuse on the course has reduced during that time, but more needs to be done to improve participation levels.

"I think things are changing for the better but you're never going to eradicate it and I don't think Sergio's comments help," he said. "The old twitching of curtains and people thinking: 'What's he come for, no one's ordered a taxi or a takeaway', have gone. They don't have to worry that there will be a corner shop on every tee or their 18-hole course won't be reduced to 17 because someone's nicked one. We pay in pound coins, not rupees.

"It doesn't bother me these days. It used to but now I shrug my shoulders and think if people don't want my money then I'll go and play somewhere that they do.

"The governing bodies are concentrating on things that are important to them, and maybe the 2% aren't as important as sponsorship. They need to be more engaging and take people's advice. We go into schools and try to break down the perceptions of golf clubs to kids, and change the perceptions of the kids because they think it's a white middle-class game that's unattainable to them.

"The wealthy clubs can pick and choose the members that they want, not just if they are black and white. But Tiger Woods, for all his faults, has made golf cool and brought it to the masses. Sergio saying what he says, that doesn't help and doesn't do golf any good."

In recent years the issue of racism has caused huge controversy in football after two high-profile incidents. The García-Woods incident now threatens to cause a similar furore, but the more serious concerns lie at grassroots level.

The 2% participation figure, which is made up of participants over 16, compares with 29.4% in cricket and 18.3% in football. However, Brendan Pyle, the development manager at the Golf Foundation charity that encourages participation for young people in deprived areas, insists work is being done to improve the problem. "We've had a focus on city areas in recent years. We're developing a new way of playing golf away from the golf club which is known as street golf, which uses a safe ball which will travel approximately 100 yards and means we can take the game into disadvantaged and underprivileged areas."

Teaching ethnic minority youngsters away from the surroundings of a club may be one way to improve participation levels. But changing perceptions that golf is a sport for only the elite is a long battle that requires much more attention.

"When we do coaching sessions in Bradford's inner-city schools we don't say: 'Oh, you're white, you can't play'," said Athwal. "Next time the governing bodies are at their board meeting, have a look at how many black faces are around the table – that's exclusivity."