On the way to collect his Nobel prize just before Christmas, the physicist Peter Higgs told me: "When I was working on my 1964 paper [that first predicted the existence of the "boson" named after him], nobody else took what I was doing seriously – nobody would have wanted to work with me." And yet, 50 years on, who hasn't heard of this subatomic bundle of energy claimed to be the last piece of the physics jigsaw – the existence of which was confirmed at the Large Hadron Collider in 2012?

Higgs could certainly not have anticipated the huge scientific endeavour to build and run the LHC at Cern, the European nuclear research organisation – a project that was required to confirm his prediction and would become an obsession for many thousands of the world's smartest scientists and engineers, happy to dedicate their careers to the project. Frustratingly, the argument about the importance of investment in basic scientific research like that carried out at the LHC, and its proven long-term value for economic growth and prosperity, is still having to be made: most physicists are bored with having to remind the world that the web was invented at Cern. But surely no one would, in hindsight, begrudge the minuscule amount of taxpayers' money that paid for Higgs's salary at the University of Edinburgh all those years ago.

And yet in today's climate of harsh realities and impact-obsessed purse-string holders, Higgs would have been unlikely to receive any funding to conduct his research – for he was something of a maverick who worked alone in an unfashionable area of speculative theoretical physics. While lip service is still paid to the importance of funding basic research that does not have any obvious or immediate application in industry or societal benefits, Higgs would struggle to hang on to his academic post today. You might think that someone like him really need publish only one or two papers of (eventually vindicated) Nobel-worthy research over his entire career, but in today's "publish or perish" climate, that would simply not cut it.

The research output of every academic in UK universities is currently being assessed – measured by the quality of four papers published in peer-reviewed journals – as part of the research excellence framework. This hugely time-consuming exercise takes place every five or six years with the intention of "providing accountability for public investment in research". I am just one of an army of academics tasked with evaluating the research of our peers and its oh-so-necessary impact. In fact, mention the word "impact" to most academic researchers in Britain and watch their reaction of frustration and exasperation. But it's a game we all have to play.

From 2015, UK funding bodies (not just of science and engineering) will use the assessments to inform the allocation of research money to all university departments. It is true of course that this exercise provides accountability for public investment and requires evidence of the benefits. But the system clearly discourages academics from embarking on speculative basic research, that in Higgs's case took 50 years to vindicate – and even then without showing impact beyond inspiring the masses with the wonders of particle physics.

Encouragingly, the biggest ever EU research and innovation programme, called Horizon 2020, recently launched in the UK, with €80bn of funding available. The bulk of the funding will be focused on applied science, in such areas as climate change, energy, security, health and transport, but the word is that Horizon 2020 will also put aside a significant proportion for basic, blue-sky research.

The worry is that However, if we cannot break the government's obsession with impacts, measurable targets and almost immediate applications, not only will we be closing the door on any future Nobel prizes such as the one awarded to Higgs, but we will be losing our grip on the Haldane principle (that decisions about research spending should be made by researchers, not politicians) – and the UK would certainly lose its place among the leaders in world science. There are plenty of other nations queueing to replace us.

• Jim Al-Khalili's new series of The Life Scientific, on Radio 4, starts on 18 February with Peter Higgs as the guest