Ben Goldacre, The Guardian, Saturday 22 May 2010

So all good citizens this week are poring over the “Programme For Government”, and it’s true to say that there is much to be pleased with. Labour wasn’t all about unbridled credit and fun public sector spending sprees: they kept all your emails, kept records of the websites you visited, used “anti-terrorism” legislation on people who plainly weren’t terrorists, and so on.

But most interesting are the noises now being made about crime and evidence. “We will conduct a full review of sentencing policy” they say: “to ensure that it is effective in deterring crime, protecting the public, punishing offenders and cutting reoffending. In particular, we will ensure that sentencing for drug use helps offenders come off drugs.”

These are grand promises. Compulsory addiction rehabilitation with “Drug Testing and Treatment Orders” was introduced precisely 10 years ago as an alternative to custodial sentences or simple probation for those who have committed drug related crimes. Their implementation without adequate analysis represents just one particularly graphic example of our tragic ineptitude at running simple trials of social policy.

A judge making a decision on a criminal’s sentence is in the exact same position as a doctor making a decision on a patient’s treatment: they are choosing an intervention for an individual in front of them, with the intention of producing a particular set of positive outcomes (reduced crime, and reduced drug use); they get through a large number of individuals in a month; and in many important situations they don’t yet know what works.

If you randomly assign a fairly large number of criminals, or patients, to one of two interventions, in situations where you don’t know which intervention would be best, and measure how well they’re doing a year or so later, you instantly discover which intervention is best. Add in the cost and you know which is most cost effective. The basic principles behind this idea were first described in Daniel 1.12 of the old testament, and can hardly be considered new.

Before being rolled out nationally in October 2000 DTTO’s were extensively piloted in 3 cities by the Criminal Policy Research Unit of South Bank university, at considerable cost. What insights did this generate? There was no randomisation, and no “control” group of identical criminals given traditional sentences for comparison, so the only new knowledge generated by these pilots was the revelation that it is possible to set up a DTTO service and run it in some buildings in some cities.

As it happens, when they did follow up the people who had passed through the service, they hadn’t done particularly well, but this finding wasn’t published until after the service had been rolled out, and in any case, the chosen study design means we have no idea how these participants would have turned out if given a traditional custodial sentence, so there’s no sense in giving these results even a moment’s thought.

This is a tragedy, and not just because drug use is estimated – with the usual caveats on estimating nebulous notions – to cause 85% of shoplifting, 80% of domestic burglaries, over half of all robberies, and so on. This is a tragedy because it speaks to motives that will never go away.

It takes a brave and modern politician to say “well, I want to introduce a new policy, but I honestly don’t know if it will work”, to try it out on half of a group of people, and then measure their outcomes a few years later, perhaps long after the politician has moved on. This would revolutionise social policy, in far wider domains than sentencing policy, and if Cameron and Clegg were really young and visionary, they would step up to the plate. But politicians – we must share the blame – will always go further when they reason in terms of rhetoric and absolutes.