It was karaoke night at the Weaver's Arms when Chantelle Richardson was attacked by a stranger because of her disability. The 23-year-old, whose face has been disfigured since she was 14, had already left one pub that night after comments about her condition. Now drinking with mates at her local in Oldham, it happened again.

"Is your friend wearing a mask?" said the woman who'd just stopped singing as one of Richardson's pals approached the mike. "Your friend's face is disgusting." The woman repeatedly told Richardson: "Take off your mask," before punching her in the face. The blow was so strong it could have been fatal and left Richardson hospitalised for weeks. For months, she was depressed and afraid to go out in public. Her attacker, brought to court in March last year, was sentenced to eight months in prison.

Richardson's story is shocking, but it is not unique. Disability hate crimes are under-reported by the media, and under-investigated by the police. But it's a problem that is getting worse not just behind closed doors, as happened at the private care home Winterbourne View, near Bristol, but on streets and in public spaces across Britain.

In a fortnight London hosts the Paralympics where the sporting achievements of more than a thousand disabled athletes will be rightly celebrated. But beyond the Olympic Park a more unpalatable view of how Britain treats people with disabilities exists, where disability hate crime is at its highest level ever recorded. In 2011, there were close on 2,000 recorded disability hate crimes in England and Wales, double the number in 2008 when records began.

Deteriorating situation

In March, the Home Office estimated that 65,000 disability hate crimes occur each year. And disability charities say it could be as high as 100,000. They have little doubt that the deteriorating situation is being driven by "benefit scrounger" abuse.

The Department for Work and Pensions has been accused of irresponsible rhetoric, in particular for its suggestion that three in four people claiming incapacity benefit are faking disabilities. It now estimates that only 0.3% of the incapacity benefit budget is overspent due to fraud. A DWP spokesman says it is "absolutely committed to supporting disabled people", but he acknowledges that "we need to work together and do more to change negative attitudes".

Yet Katharine Quarmby, author of Scapegoat: Why We Are Failing Disabled People, warns: "Unless the government decouples reducing benefits from hinting strongly that most disabled people are scroungers, then we're going to see more attacks."

That is what happened to Emma Round, 28. In 2009, nerves in Round's abdominal cavity became badly damaged, making walking extremely painful. Unable to leave the house without help, she became effectively trapped for nine months until a doctor decided to give her a wheelchair. The change revolutionised her independence, but also made her a target of abuse.

Sitting outside a local shop not long after getting her wheelchair, she was approached by a well-dressed stranger. "He lent over, looked me in the eye and said, 'It's a good scam. Someone your age shouldn't be in a wheelchair ... You're just doing it for the benefits, aren't you? Scum.' Then he walked off. I just sat there crying," Round says.

That incident was not a one off. "Since then I've been called lazy frequently and had people grab my wheelchair and tell me to get out and walk. I've been called a scrounger, a sponger, a faker and other words you wouldn't be able to put in print."

Round's story is replicated every day across the UK. Disability charities receive a constant flow of reports – accident victims spat on and punched, wheelchair users pushed into oncoming traffic, injured army veterans told they are exaggerating their injuries.

A closer look at last year's crime figures, obtained, under freedom of information requests, for every police force in England and Wales, and published by the Guardian today in full for the first time online, reveals the gap between reality and how many incidents are being recorded and lead to convictions. Only 1,942 of the estimated 65,000-100,000 disability hate crimes were recorded in 2011, and just 523 people were convicted in 2011.

In Lincolnshire and Warwickshire, two counties with a combined population of 1.2 million people, not a single person was convicted of disability hate crime last year. In half of the 42 police authorities in England and Wales, the number of guilty offenders did not reach double figures. This is in contrast to the 10,000-odd people successfully prosecuted each year for racial hate crimes.

Hidden scandal

The underlying causes for the gap are complex and often obscured, but charities agree that the failure of the criminal justice system lies at the heart of this hidden scandal. Campaigners had hoped the case of Fiona Pilkington, who killed her disabled daughter and then herself in 2007 after police failed to heed calls for help after years of harassment from local youths, would be a turning point in the policing of disability hate crime.

Police have been required to collect and report instance of disability hate crime in a standardised way since 2008. But Clive and Heidi Cox certainly feel the system has failed them. Back in June, during the rainy jubilee weekend, they mourned the third anniversary of the death of their son David, who was autistic. David, 18, was killed by a punch to the face after getting into a scrap with a gang of youths. His parents claim that it was a disability hate crime. South Yorkshire police disagree. Three years on, no one has been prosecuted.

David was diagnosed at the age of 11 with Asperger's syndrome and mild attention deficit hyperactivity disorder. "He was cheeky," Heidi recalls. "He'd probably want to be the centre of attention, but sometimes he'd go about it the wrong way." That naughty streak kept him out of full-time education for much of his childhood. By his late teens, the family decided to move to Doncaster in the hope of a fresh start.

At first, the switch was a success – "the best 18 months of his life", according to Heidi – but then came the incident on 1 June 2009. Returning home from a kickabout with his brother, David walked past a group of local lads outside a shop and asked for a header. "Fuck off, you freak," one of the boys allegedly said, later calling him a "weirdo". The scuffle that ensued saw David receive a punch to the jaw so hard that it tore an artery and caused a brain haemorrhage. He collapsed and was rushed to hospital. At 3.47pm the following day his life support machine was switched off.

The origins of the incident remain contested – South Yorkshire police insist there was an element of self-defence in the boys' behaviour and David may have been an "aggressor". But the Cox family feel that the five 18- and 19-year-olds targeted David's disability. "They called him a freak and a weirdo," Heidi says. "Why would that boy choose those names?"

Three years on, the Cox family are still pursuing the case and have written to the prime minister, David Cameron. But coming to the end of a long line of appeals, they are left only with the gnawing feeling that justice has not been done.

It is the same conclusion – that the justice system still treats people with disabilities differently – which others have come to. In particular, there is a problem around the perceived unreliability of evidence given by people with learning disabilities, be it victims or witnesses. According to a Mencap spokesman, if a lawyer thinks that a disabled witness will underperform on the stand, they will not take the case to court.

Last month, the government ratified most of the recommendations of an inquiry by the Equality and Human Rights Commission into disability-related harassment. The coalition promised to oversee the creation of more accessible justice, better frontline training and stronger accountability from administrators. But there was a line in the response that dismayed some charities. "Ownership for tackling disability-related harassment and hate crime is primarily the responsibility of the local authorities," it read. "Central government's role is not to be prescriptive but to lead, advise and encourage." To campaigners, it had the familiar ring of responsibility being shirked.

It has now been more than a year since Richardson's attacker was sent to prison for punching and abusing her. In one important way, her story is unusual – we know what happened to Richardson. For most disability hate crime victims, their suffering remains hidden.

• This article was amended on 15 August 2012. The original said that in 2001, there were close on 2,000 recorded disability hate crimes in England and Wales. The date has been corrected to 2011. The article was further amended on 17 August 2012. The original said between one in 50 and one in 100 people claiming incapacity benefit were faking disabilities. The Department for Work and Pensions' preliminary estimates for 2010-11 suggest the figure is much lower and that only 0.3% of incapacity benefit spending is overpaid because of fraud.