The Shape of Jobs to Come with its headline-grabbing list of exotic new roles in science was based on flimsy research, says nanoscientist James Hayton

It's a big issue: how to stimulate interest in science and inspire the next generation to follow a scientific career. With the aim of doing just that, the government-backed Science: [So what? So everything] campaign last month released a list of 20 possible future jobs, based upon expected advances in science and technology.

The exotic list, from "body-part maker" to "space architect", generated huge media interest, and was publicly endorsed by prime minister Gordon Brown, science minister Lord Drayson, and Stephen Fry, who supports the Science: So What campaign.

The list was based on a lengthy commissioned report, entitled The Shape of Jobs to Come, researched and written by a futurist consultancy called Fast Future. Unfortunately, despite the best intentions of Science: So what? (SSW), there were major flaws in Fast Future's research.

My own attention was caught by the job of "nano-medic" in the list of 20 future jobs. The description claimed that "sub-atomic devices and treatments could transform personal healthcare so we would need a new breed of nano medicine specialists to administer these treatments." It is, and forever will be, impossible to make devices smaller than an atom.

Although this mistake has since been acknowledged and partially corrected, trying to trace the source of this job description revealed a huge amount about Fast Future's practices. For a whole page of information about nano-medicine, there is a single referenced source: a web page whose only mention of nano-medicine is the phrase "Nano-sized machines to deliver health. 'Nuff said."

This is indicative of the general quality of sources used in Fast Future's research. For the job of "memory augmentation surgeon", the sole source is an IT consultancy company – no reference to any neurological research into the physical basis of memory itself. Virtual clutter organisers? The only cited reference is a named head and neck surgeon who suggests the role. The job already exists anyway, but the use of informal suggestions as sole sources of information is pretty poor practice in any field of research.

Rohit Talwar, CEO of Fast Future and co-author of the report, gave the following explanation for the consultancy's choice of source information (you can see his full response here):

"[A] key approach in futures research is to look for 'weak signals'. Citing websites, news reports and blogs is an accepted best practice in horizon scanning where you are looking for these weak signals of what's coming next. These weak signals often pre-date any hard academic literature, which is why futures work deliberately draws on a range of sources not just published academic literature. In many cases there may only be one source or opinion from which you are working."

This is not the case for the jobs listed. Taking nano-medicine as an example, there are thousands of research papers and quality media reports about the role. In this instance, Fast Future is working from only one source of information not out of necessity, but out of choice. It could easily have referred to academic review papers, which provide expert summaries of peer-reviewed research, or consulted directly with scientists themselves.

It didn't, and the result is a report lacking in any real insight, because Fast Future doesn't seem to have looked at the subjects in any detail.

Worse still, where there is relevant information in the cited sources, it has been copied and pasted directly into the job descriptions that were the basis of all the publicity. Take this description of quarantine enforcer, a job that is unlikely to inspire kids to take up science and would probably never exist as a specialised career anyway:

"If a deadly virus starts spreading rapidly, few countries, and few people, will be prepared. Nurses will be in short supply. And as death rates rise, and neighborhoods are shut down, someone will have to guard the gates."

Compare this to the source, an article on Forbes magazine's website:

"If a deadly virus starts spreading rapidly, few countries, and few people, will be prepared. That'll be good for certain occupations. Nurses will be in short supply. And when people really start dying, and neighborhoods are shut down, someone will have to guard the gates."

This is not the only example of cut-and-paste. Reading the press release, or SSW's page, wouldn't it be reasonable to assume that the descriptions are Fast Future's own work, based upon extensive research from a number of reliable sources?

Independently of my own articles on this issue on the 10minus9 blog, EvidenceMatters put together a detailed and highly critical analysis of the future jobs report, revealing yet more flaws in Fast Future's methods. Meanwhile, Holford Watch asked the Department for Business, Innovation and Skills, which funds the SSW campaign, for a response to the various concerns raised. It stands by the report:

"This work is a speculative look to the future and does not inform policy or impact on government funding decisions. The report was checked by the Department for Business, Innovation and Skills and felt to be rigorous and credible for the purposes we required."

This standard of rigour and credibility shows a great deal about the department's attitude towards the promotion of science. If the purpose of the report was to inspire the next generation of scientists, or even to stimulate interest in science, it is essential to look at some of the incredible work being done by scientists and the real benefits we might gain. If promoting science is important, then so is the quality of the material commissioned to this end.

Personally, I find it deeply disappointing and frustrating that public money has been used to cobble together a report in such a sloppy manner. "That'll do" just isn't good enough.

James Hayton writes the 10minus9 nanoscience and nanotechnology blog