Innovation is like motherhood and apple pie, in that everyone's in favour of it. Especially governments. They all want more of it. The only problem is that they haven't the faintest idea of how to make it happen and so are suckers for anyone with a Big Idea about it.

A few years ago, an economist named Richard Florida found himself the beneficiary of this syndrome. He published a book with the beguiling title of The Rise of the Creative Class and the even more beguiling subtitle "… and how it's transforming work, leisure, community, and everyday life". Florida's argument was that, as our economies are being transformed into knowledge economies, creativity is to our century what access to natural resources was to the 18th. Creative occupations – an occupational category comprising knowledge-workers, intellectuals and artists – were growing to the point where almost a third of the US workforce could be regarded as "creative" and companies, cities and regions were bending over backwards to attract them.

river

Once, the fate of great industrial centres was determined by geography – their proximity to natural resources, navigable rivers, harbours, etc. But the "resources" of the new economy are footloose. The message of Florida's book for urban planners and municipal authorities, therefore, was that if cities want to succeed they must attract the creative types. In order to do so, they must cherish the three "Ts" – talent (a highly educated population), tolerance (cultural and sexual diversity) and technology (the infrastructure necessary to sustain an entrepreneurial culture). In other words, they should look a bit like San Francisco.

As it happens, some of Florida's insights were really just contemporary articulations of the ideas of ancient economists. Alfred Marshall observed many years ago that workers in skilled occupations tend to cluster in areas where their peers live and work. But why did those people choose to settle there in the first place?

Which brings us to Cambridge, Marshall's alma mater and the nearest thing Europe has to Silicon Valley. Inevitably, it has been dubbed Silicon Fen. The "Cambridge phenomenon" – the extraordinary ecosystem of science- and technology-based companies in and around the town – has acquired near-mythological status. At the moment there are something like 1,500 of these companies in the region, some of which have become market leaders in the UK and the wider world. Five of them have valuations in the billion-dollar region, and one, ARM, is one of the UK's most successful companies and the dominant firm in one of the fastest-growing markets in the world: that for the processor chips that power smartphones and other mobile devices.

The Cambridge phenomenon has functioned as a honeypot attracting venture capitalists, big consultancy firms, bankers and other specialist organisations that attend to the needs of growing firms in complex industries. Cambridge is now ranked as one of the top three "innovation ecosystems" in the world, according to a recent international survey. It is where Microsoft chose to set up its major European research lab. Ditto Toshiba. And it's where AstraZeneca has chosen to locate its global R&D and corporate headquarters, a decision that will involve spending upwards of £300m and bring about 2,000 researchers and support staff.

sanger

Just down the road, in the village of Hinxton, is the Genome Campus, at the heart of which sits the Sanger Institute, named after the only British scientist to win two Nobel prizes, where Sir John Sulston and his colleagues first sequenced the human genome. And not far away is the Babraham Research Campus, which specialises in the incubation of bioscience companies and into which the government recently injected £44m of public funding.

For decades, policy-makers from Europe and beyond have been coming to Cambridge, wistfully eyeing its "technology cluster", shaking their heads at the list of vibrant companies that it has spawned, interviewing academics, executives, planners and venture capitalists and wondering what its secret formula is.

There is no formula. Nobody planned the Cambridge phenomenon, just as nobody planned Silicon Valley. Both developed organically. That doesn't mean that similar phenomena can't happen elsewhere, just that they can't be delivered to order. And they take lots of time to evolve and mature.

In Silicon Valley's case, the story goes back to 1939, when two Stanford-trained engineers, David Packard and William Hewlett, set up a little electronics company in a Palo Alto garage. In the case of Cambridge it also goes back a long way. Because of lobbying by the university in the early part of the 20th century, the town had no "legacy" industries – no mass production, no car manufacturing (unlike Oxford), no smokestacks. Insofar as Cambridge had any industrial sector at all in the first half of the 20th century, it was in relatively clean areas such as consumer and broadcasting technology (Pye, founded in 1896) and scientific instrumentation (Cambridge Scientific Instruments, co-founded by Charles Darwin's fifth son, Horace, in 1881).

The first stirrings of change came in the early 1950s, when Cambridge University embarked on the growth path that has turned it into one of the world's intellectual powerhouses. In 1949, in what was then called the Mathematical Laboratory (and eventually became the Computer Laboratory), Maurice Wilkes and his colleagues had built the Edsac, one of the first general-purpose electronic computers. Unlike other researchers who were building similar machines in other places, Wilkes & Co made theirs immediately available as a computing tool to colleagues in a range of disciplines. This had two effects: it led to a series of scientific breakthroughs (for example, plate tectonics in earth sciences); and it established Cambridge as a leader in information technology and computing.

A second source of innovation was the discovery in 1953 by Watson and Crick of the structure of the DNA molecule. This was the first link in a chain that led to the sequencing of the human genome in 2000 – and eventually spawned a whole ecosystem of biotechnology companies.

Arm

A third significant development was the founding, in 1960, of Cambridge Consultants, one of the UK's first technology-transfer companies. It was set up by three Cambridge alumni to "put the brains of Cambridge University at the disposal of the problems of British industry". As far as the IT segment of the ecosystem is concerned, Cambridge Consultants played a critical role, because one can trace links from many of today's successful companies (for example ARM) that stretch back to it.

A fourth factor was the change that took place in the 1960s in the university's and the local authority's attitude to industrial development in the town. Within the university, much of the pressure came from researchers in physics, engineering and computing who came to see local industrial development in their fields as desirable for various reasons: as a way of exploiting their research via startup ventures; as a potential source of research collaboration and funding; and as a way of boosting the employability of their students. These pressures were amplified by the university's liberal – some would say relaxed – attitude to intellectual property.

On the planners' side, there were pressures from central government to reverse their previous anti-industrial bias for the area. A particular cause célèbre had been their refusal to permit IBM to locate its European R&D laboratory in Cambridge. In the end, the planners changed their minds after a commission led by Sir Nevill Mott, head of the Cavendish Laboratory, signalled a radical shift in the university's attitude to industrial development. From that moment, the die was cast.

What we see in Cambridge and Silicon Valley today are complex industrial ecosystems. But while a formula is unattainable, we can see that such ecosystems need certain key ingredients if they are to thrive.

First of all, the ecosystem needs to have a significant research university at its heart. It has to be in a pleasant urban environment that has a good housing stock, reasonable transport links, good state and private schools, good healthcare and a lively cultural life. And the local planners have to be sympathetic to the needs of small tech-based companies, especially in their early stages of their corporate life.

Cambridge

It's important that the university at the heart of the ecosystem should be one that gives considerable freedom to its academics and has a liberal attitude towards intellectual property. Also needed are legal firms that understand IP, and outposts of the global accounting and consulting firms because venture capitalists do not feel comfortable funding ventures which are audited by local firms. And there even needs to be a range of different kinds of hotel – including boutique ones – to cater for visiting researchers, financiers and conference attendees.

The ecosystem needs a lot of small specialist firms that can quickly tackle specific commissions together with a networking system that enables people quickly to locate a particular specialist when they need him or her. Most important of all, it needs a significant population of "angel" investors – experienced entrepreneurs who have acquired wealth through building successful companies and who are willing and able to invest in, and mentor, firms at the startup stage, long before any venture capitalist would deign to look at them.

Cambridge has a club of such angel investors. Membership requirements include a net worth in excess of £15m and a track record of at least one successful "exit" from a startup. At present, the club has 56 members. It functions as an arena in which innovators and entrepreneurs can pitch ideas to an audience that is experienced, sympathetic but critical and exceedingly well-informed. Not every pitch that succeeds in this forum results in a successful new company. But it provides a useful reality check on scientific and technological dreams.

At another level, the "Cambridge phenomenon" tells us that innovation ecosystems cannot be bought off the shelf and installed wherever governments wish to locate them. Innovation is a complex and delicate plant. You have to prepare the soil carefully, be prepared to wait for the fruits and accept that your administration will be long gone before they materialise. Even in quickly moving technologies, patience is a virtue.