Let’s run this one up the flagpole and see who salutes. The business world has its own approaches to trying out new ideas, and its own distinctive idioms for doing so. Public services pride themselves on more considered, scientific ways of establishing what works when it comes to spending the taxpayer pound.

But as the most ambitious benefit reform of the modern era gets the go-ahead after only limited piloting, it’s worth asking if such pride is really justified.

Universal credit is this week starting to be extended nationally after what ministers describe as “remarkable” results in pilot areas. Read the small print of the evaluations, though, and you’ll find heavy qualification such as “interim findings”, “useful insights” and “this analysis only considers the impact on new claims during the very early stages of the policy process in a small number of offices”. There’s a clear sense of backsides being covered.

Over in health, meanwhile, more than 100 expressions of interest are thought to have been lodged by last week’s deadline in establishing the first “new models of care” that are expected to cover more than half the English system by the end of the decade under NHS England’s Five Year Forward View.

How will these models be assessed and what route will there be from vanguard to preferred option – or failed concept?

Some of the bidders to become so-called “vanguard” projects, sharing an initial £200m funding in 2015-16, want to be the proposed “multi-specialty community providers”, merged or partner GP practices providing a range of services including community hospitals. Others aspire to be the alternative “primary and acute care systems”, hospital trusts transforming into integrated care organisations that also offer services in a range of settings. Setting aside for now the fact that this seems to be teeing up another acute-versus-primary care clash, something that has bedeviled the NHS since 1948, The immediate questions are how these models will be assessed and what route there will be from vanguard to preferred option – or, you never know, failed concept.

While the utility of policy-piloting is largely taken for granted in government, there is no single template for how it should be done. Universal credit was tested first in just four “pathfinder” sites in the north-west, mainly among jobless single claimants. One of the evaluation studies cited by ministers was based on the views of 900 claimants put on the new benefit.

In a fascinating paper in the Journal of Social Policy, a team from the faculty of public health and policy at the London School of Hygiene and Tropical Medicine (LSHTM) analyses the piloting of three policies under the last government: preventive services to avoid hospital admission under the Partnerships for Older People Pilots (Popps) banner; individual budgets for social care users; and telehealth and telecare under the Whole System Demonstrator (WSD) programme.

NHS England is avoiding the term 'pilot' for its new models of care, preferring 'early adopters' and 'first cohort'

All three pilot schemes had multiple, and changing, purposes, the team concludes. The Popps pilots began as a means of learning from local experience of a variety of interventions, but were later required to try to prove general effectiveness. The WSD was set up to test the effectiveness of assistive technology, but also to demonstrate how it could be used successfully and to diffuse its application.

The individual budgets pilots were designed both to assess the effectiveness and cost-effectiveness of giving people their own care budgets and to analyse barriers to implementation. But less than a year into the study, the then care services minister, Ivan Lewis, announced that (renamed) personal budgets would become national policy regardless – something that “hugely undermined” the credibility of the exercise, says the paper, and shifted its purpose from experimentation to early implementation.

Based on the three schemes, the LSHTM team says that policy piloting by the UK government, or at least the Department of Health, may be seen as something that “primarily serves purposes other than generating evidence of ‘what works’”.

In fairness, NHS England is studiously avoiding use of the term “pilot” in relation to its new models of care programme, preferring vanguard, “early adopters” and “first cohort”. With bidders having been told they must have “a credible plan to move at serious pace to make rapid change in 2015”, the intent seems clear. But it would still be good to know that it works.



