Before her grandson's condition became clear, Margaret Palmer generally thought the best of people. Now she has a very different view.

"The cruelty, the unkindness, the lack of understanding towards people with autism is horrifying. Not just police and social services, but people, the public," she said. "Adrian was lovely, a fantastic lad and so special to us. It's the way other people treated him that was the problem."

A new survey carried out by the National Autistic Society (NAS) and to be published this week to mark the launch on Monday of its new campaign, Careless, shows just how hard life can be for adults with the disability. A startling 44% of those questioned admitted they stayed indoors as much as possible for fear of being harassed. Almost a third reported having had money or possessions stolen, while 37% had been forced or manipulated into doing something they didn't want to do by someone they thought of as a friend. Almost half (49%) of the 1,300 people surveyed reported having been abused by someone they thought of as a friend.

A consultation is under way on the government's new criteria for eligibility for adults to access social care, and the NAS is deeply concerned that even as numbers of people being diagnosed with autism disorders rise, the legislation will see levels of support and care drop.

"It's already incredibly patchy," said Tom Madders, NAS campaign manager. "Only half of local authorities even offer a pathway to diagnosis. We're starting to pick up kids who have autism a little better but for adults services are too few and far between. We hear a lot about harassment and isolation, about people being arrested when they haven't committed a crime, but these survey results, the abuse and the neglect, still shocked us," he said.

"People with autism can find it difficult to interpret others' motivations, misjudge relationships, and left unsupported many are taken advantage of. Our key concern is that government plans for the care system could make this desperate situation even worse. The new criteria take away entitlement of support for people if 'abuse has occurred or will occur', but that is an essential criteria for vulnerable people with autism."

Adrian Palmer was diagnosed with Asperger's syndrome at 14 and assessed for support by social services at 18. His family knew he was at risk from the people he was spending time with and needed help with the transition to adulthood. But although the assessment found Adrian struggled with interpreting people's motives and was easily led, the risks were not sufficient for him to receive support.

At 21, he reported being raped by a man who had befriended him. His parents appealed again to the police and social services and finally a small amount of evening support was approved. Five days later in May 2006 Adrian was killed by the man he had accused of rape. Under the government's proposed criteria, Adrian would not have been eligible for support. West Mercia police were later found to have failed him by not properly investigating his complaints and social services apologised to his family.

His killer, Ben Murphy, 23, was sentenced to four years in prison for manslaughter and served two.

"It's horrible to think what just a little bit of support might have done, it might have changed everything," said Margaret Palmer. "You try your best but you need some support sometimes, not every family does but some do. We did. But the police do nothing, the social services do nothing.

"Adrian's was a long, long story. He went through so much from when he was young, all through school people were horrible to him, teachers and pupils. It was awful to take him to school, his little heart pumping, so anxious. It would upset us terribly that no one would listen or try to understand that if you just treated him right he would blossom," she added.

"When he was 18 he started to drink. We tried to say it's not the way to be sociable, to make friends, but social services just said he'll learn by his own mistakes. They advised us to make him homeless if we couldn't cope. I was shocked. People he was mixing with were horrible, but he couldn't see he was being manipulated."

Another sufferer, Rose, 54, was diagnosed with autism 18 months ago. Her condition only became obvious when her husband left five years ago, forcing her to fend for herself. Her daughter, now 21, was diagnosed aged 19 and her son when he was nine. "It makes you very prone to depression," she said. "All of us are now on medication. It makes you feel worthless when you know you don't fit in. You feel like you're out on a limb." She has been targeted by people who have stolen from her and sexually abused her.

"It's hard when you don't know someone's agenda. You just want to make friends but you feel out of your depth. I have battled hard to get any help at all. With my experiences over the last few years I have grown to mistrust people totally. It doesn't seem there are many decent people left."