Evidence-based policymaking is now central to the scientific agenda: most researchers need to demonstrate that they are making an impact on policy, and want to help bridge the evidence-policy gap. Ongoing debates over energy policy are one of many examples in which scientists bemoan a tendency for policymakers to produce ideological rather than ‘evidence based’ decisions, and do their best to change their minds.

Yet these efforts will fail if scientists and other experts fail to understand how the policy process works. To do so requires us to reject two romantic notions: first, that policymakers will ever think like scientists; and second, that there is a clearly identifiable point of decision at which scientists can contribute evidence to make a demonstrable impact.

To better understand how policymakers think, we need a full account of “bounded rationality.” This phrase describes the fact that policymakers can only gather limited information before they make decisions quickly. They will have made a choice before you have a chance to say “more research is needed”! To do so, they use two short cuts: rational ways to gather quickly the best evidence on solutions to meet their goals; and irrational ways - including drawing on emotions and gut feeling - to identify problems even more quickly.

This highlights a potential flaw in academic strategies. The most common response to bounded rationality in scientific articles is to focus on the supply of evidence: to develop a hierarchy of evidence, which often privileges randomised control trials; to generate knowledge; and to present it in a form that is understandable to policymakers.

We need to pay more attention to the demand for evidence, taking more account of lurches of policymaker attention, often driven by quick and emotional decisions. For example, there is no point in taking the time to make evidence-based solutions easier to understand if policymakers are no longer interested. Successful advocates recognise the value of emotional appeals and simple stories to draw attention to a problem.

To identify when and how to contribute evidence, we need to understand the complicated environment in which policymaking takes place. There is no “policy cycle” in which to inject scientific evidence at the point of decision. Rather, the policy process is messy and often unpredictable. It is a complex system in which the same injection of evidence can have no effect, or a major effect.

This system contains many actors presenting evidence to influence policymakers at many levels of government; networks, which are often close-knit and difficult to access because bureaucracies have operating procedures that favour particular sources of evidence over others; and a language within policymaking institutions indicating what types of thinking have greatest traction (such as “value for money”). Social or economic crises can prompt lurches of attention from one issue to another, or prompt policymakers to change completely the ways in which they understand a policy problem. However, changes to well-established ways of thinking in government are rare, or take place only over longer timeframes.

This highlights a second potential flaw in academic strategies: the idea that the impact of research can be described as a set-piece event, separable from the policy process as a whole. Instead, we need to focus on long-term strategies: investing the time to find out where the action is, and how to exert influence as part of a coalition of like-minded actors looking for opportunities to raise attention to problems and solutions.

Unfortunately, these insights mostly help us identify what not to do. And the alternatives may be difficult to accept (how many scientists would be comfortable making manipulative or emotional appeals to generate attention for their research?) or deliver (who has the time to conduct research and seek meaningful influence?). However, only by engaging with the practical and ethical dilemmas that the policy process creates for advocates of evidence, can we produce strategies that are better suited to a complex real world.

Paul Cairney (@CairneyPaul) is professor of politics and public policy at the University of Stirling p.a.cairney@stir.ac.uk. This post is based on his new book The Politics of Evidence Based Policymaking, which was launched this week by the Alliance for Useful Evidence. More details are available on his website.