The chief inspector of prisons has described unprecedented levels of violence, bullying, overcrowding and the highest level of suicide in 10 years in a detailed examination of the way inmates are incarcerated in England and Wales.

In his 100-page annual report, Nick Hardwick highlighted how safety outcomes in prisons had declined significantly for inmates, particularly the most vulnerable, and a third of prisons inspected were deemed not safe enough in 2013-14.

The most dramatic example of this lack of safety was the 69% increase in self-inflicted deaths in the financial year 2013-14, to 88. This was the highest figure in a decade. As the Guardian reported last week, the number of suicides is continuing to rise – between January 2013 and 2 October 2014, 134 adults, including four women, killed themselves in prison; a rate of more than six deaths a month.

Hardwick said the rise in suicides, violence and self-harm, could not be attributed to a single cause. But, he added: “In my view it is impossible to avoid the conclusion that the conjunction of resource, population and policy pressures, particularly in the second half of 2013-14, and particularly in adult male prisons, was a very significant factor.”

Among the themes Hardwick picked out as contributing to the rate of suicide and self-harm were:

• A failure to correctly identify risks for new prisoners when they arrived and were often at their most vulnerable.

• Poor mental health provision; a third of those who had taken their own life were on the assessment, care in custody and teamwork (ACCT) system, which is used to support the most vulnerable inmates.

• The inappropriate use of segregation, with too many inmates in “crisis” being segregated and living in poor conditions.

• A failure to heed recommendations from the prisons and probation ombudsman.

• And understaffing, both from lack of resource and unfilled vacancies, contributing to the rise in self-inflicted deaths.

In adult male prisons, the number of assaults involving inmates rose by 14% year on year, and was the highest for any year for which there is data. This surge in violent attacks inside prison included what Hardwick called a dramatic 38% rise in the number of serious assaults.

Prisoner-on-prisoner attacks were one indicator of the level of bullying in prison, he said. Another was the number of incidents at height – where inmates climb on to the netting or railings on wing landings in an attempt to be sent to segregation and then moved out of the prison to somewhere safer. These incidents increased across the whole prison estate from 732 to 1281 incidents in 2013-14, including a rise from 591 to 1007 incidents in adult male prisons, and from 110 to 206 among 18- to 20-year-old men.

Amid the violence, prisoners were living in overcrowded conditions in two-thirds of the prisons inspected. “After a period of improvement, overcrowding worsened as the year progressed,” Hardwick said. Too many prisoners were spending too much time in their cells, and vocational programmes, while good where they existed, were only available in a few prisons.

On sites where there were staff shortages, violence and overcrowding, the situation could be turned around by experienced staff who created good relationships with inmates and provided positive role models, he said, citing HMP Pentonville as an example. “… beset with staff shortages, an appalling physical environment and a very needy population with high levels of substance misuse and mental health problems” the prison had some impressive staff and governors who were working “heroically” against the odds.

Hardwick said there were cases where the care provided to the mentally ill was good, but overall mental heath care was inconsistent and staff were not properly trained to identify inmates with psychological problems or to respond to them appropriately.

The report also hinted at some progress. In women’s prisons the outcomes from the inspections were all rated as reasonably good or good. Hardwick’s visit to HMP Holloway was the most positive yet for that prison, he said.

Women’s prisons were safer than men’s, and there was better mental health care, better inductions for new prisoners, and better substance misuse programmes. But women were still self-harming at disproportionately high levels, and three inmates killed themselves in women’s prisons in the financial year 2013-14.

Many fewer children were being sent to prison; the population falling from 3451 in 2002-03 to 1708 in 2012-13, and 1334 in 2013-14, a 22% drop in one year alone. But the fall meant capacity had been reduced as establishments closed and there was now a more concentrated mix of vulnerable, challenging and sometimes very violent boys being held together.

“In all establishments there were fights and assaults almost every day,” Hardwick said. At Warren Hill in Suffolk, there were 137 assaults on young people, 48 assaults on staff and 112 fights in the six months before his inspection. One member of staff was left with broken bones, concussion, stab wounds and black eyes after an attack there.

But suicides were the “most unacceptable feature” of overcrowding, understaffing and lack of safety, Hardwick said, referencing a “seminal thematic report” in 1999 by the then Chief Inspector David, now Lord Ramsbotham, on suicide, after the number rose from 68 in 1997-98 to 83 in 1998-9.

The conclusion Ramsbotham drew was that “the total experience of imprisonment” affected suicidal behaviour. The same was true today, Hardwick said. “Then, as now, it requires acknowledgement, action and accountability for doing so from top to bottom.”