A refusal by police to regard tackling antisocial behaviour as "real police work" has led to officers "retreating from the streets", according to a report today by Her Majesty's Inspectorate of Constabulary.

The study says the scale of antisocial behaviour has been widely ignored, with 14m incidents in the past year, of which only a quarter were reported to the police.

Sir Denis O'Connor, the chief inspector of constabulary, says that for 20 years police have increasingly left the public to cope with persistent antisocial behaviour. Victims have suffered repeat attacks and a high level of intimidation if they complain to the police.

O'Connor said he was concerned that spending cuts could further reduce frontline police manpower. He told BBC Radio 4's Today programme: "Over time, this has been cut back. The first thing that's gone is the front end on the streets and I think it's been a big error.

He warned chief constables that it would be a mistake to respond to spending cuts by reducing the work they do on antisocial behaviour to concentrate on crimes such as burglary and car break-ins.

"I don't think this [anti-social] behaviour is going to reduce because we are going through an austerity period. I'm concerned that the police availability, which is already low, if it is reduced further, I do not see much hope for people."

The study calls for communities to "mobilise their defences" by re-establishing acceptable rules of behaviour to deal with youths who gather in town centres, aggressive drivers in residential streets and drunks.

The report says that for victims of anti-social behaviour the experience tends to be cumulative and corrosive, but many agencies, including some police officers, see it as a mixed bag of crime, disorder and rowdy behaviour.

The HMIC estimate of 3.5m incidents reported to the police in 2009-10 is based on non-validated data drawn from the 43 police forces in England and Wales. More than 60% of the incidents were classified as rowdy or inconsiderate behaviour, with nuisance parking and problem neighbours second and third on the list.

"Despite its high public profile in recent years, antisocial behaviour does not have the same status as 'crime' for the police," says the report, adding that the public do not draw a distinction between crime and disorder: "For them it is just a sliding scale of grief."

The report says that for 20 years success for the police has been defined by cutting crime figures, while concern over antisocial behaviour has grown.

The study, undertaken with the pollsters Ipsos Mori and Cardiff University academics, says the call screening systems used by 30 of the 43 forces in England and Wales are making matters worse by downgrading complaints. It takes up to 18 months before anything effective is done even in calls that are given priority.

It says only 13 of the 43 forces have call management systems that effectively identify repeat victims who are most at risk at the time the call is made.

Too many community safety partnerships, which bring together the police, local authorities and other agencies, are bogged down in a meetings culture and far less effective than assumed in Whitehall.

O'Connor says: "Perpetrators need to know they are wrecking lives and that they will get swift action from the authorities if the public call for help."