Rory Stewart wasn’t responsible for the National Security Council Huawei leak. His move from prisons minister to the Department for International Development was not of his own doing. But it nonetheless came at a good time for the Eton-educated ex-soldier who, according to rumours, may be played by Orlando Bloom in a biopic of his life.

Stewart leaves behind a failing prison system in which violence is at record levels and, according to the most recent prison watchdog report, is struggling to deal with a drugs crisis that is “out of control”. It comes as little surprise that British prisons are failing: there’s been a 77% increase in the number of offenders locked up in the last 30 years, reoffending leading to a return to prison is rising and, according to the Prison Reform Trust, there’s soon to be a cash shortfall of £463m in the government’s investment proposals. While some working around the prison system have praised Stewart for his “energy, focus and critical thinking”, his efforts were actually failing.

We have too many prisoners, a crumbling estate, little confidence on the ground and an overuse of short prison sentences

The picture from British prisons is bleak. In the 12 months to March 2019, there were 317 deaths in prison custody. Reported self-harm incidents reached a record high of 55,598 in 2018, a 25% increase from the previous year. The number of assaults reached a record high of 34,223 in 2018 too, a increase from 2017. The fact that levels of assault dropped slightly in the final quarter of 2018 seems little cause for celebration – the same thing happened a year previously, then they increased significantly once again.

“There will be a good deal of regret at the loss of Rory Stewart from the prisons brief,” Peter Dawson, director of the Prison Reform Trust, told me, “if only because he had stayed in it long enough to begin to understand the complexities of repairing the damage done by previous administrations.” Dawson referred to Stewart’s “genuine desire” to make a difference, but the reality is clear: despite a fairly competent minister’s best efforts, the system remains broken. We have too many prisoners, a crumbling physical estate, little confidence on the ground and a penal system that vastly overuses shortsighted short sentences.

Without widespread and radical reform in our justice system this simply won’t change – but that’s why Stewart’s commitment to resign by August 2019 if there was no marked improvement was so important. “I will quit if I haven’t succeeded in 12 months in reducing the level of drugs and violence in those prisons,” Stewart told BBC Breakfast last summer, referring to 10 jails with serious problems he had singled out for investment. His targets included HMP Humber, where 29% of inmates say they’d acquired a drug problem while in custody, and Nottingham – which the Prisons & Probation Ombudsman accused of having a “history of failing to implement recommendations” from their investigations into high numbers of deaths at the prison.

Rory Stewart defends UK aid target and vows to tackle climate 'emergency' Read more

It’ll be a few months until a report comes out that will make clear how successful Stewart’s 10-prison project has been, but if the most recent stats are anything to go by (deaths in custody figures are released more regularly than self-harm and assault, and that figure is up 18 on last year), he would have been sending in a letter of resignation.

And maybe that’s exactly what was needed. Stewart has served as a diplomat around the world, been an interim governor in Iraq and taught at Harvard. His introduction of the plastic bag tax while environment minister is one of the only successful new laws we’ve seen from this government. If he had stood aside, it might well have proven that the prison brief is poisoned, and that it can’t be changed with a few hundred million pounds of investment or tinkering around the edges. He would have proved that the system is not fit for purpose.

We need to scrap sentences of under a year, and put rehabilitation – not punishment – at the centre of justice. The need to incarcerate anyone who commits a nonviolent crime should be questioned, and the causes of criminal behaviour examined so we can invest in early intervention. The profit motive should be removed completely both from prisons and probation. If Stewart, a Tory rising star, had been forced to fall on his sword, it’s possible the government would have had to engage with some of these ideas. Instead, at the time of writing, the UK is still without a prisons minister.

• Michael Segalov is an author and contributing editor at Huck magazine