Corbis

IN THE waning days of the second world war, Vannevar Bush, science adviser to President Franklin Roosevelt, penned a report that served as the blueprint for what would become America's enormously successful information-technology industry in the second half of the 20th century. With the grandiose title “Science, The Endless Frontier”, Bush (no relation to the current president) laid out a vision for government-funded science and engineering that would unite academia, industry and (this being wartime) the armed forces. This it achieved by, in effect, keeping them apart.

Under Bush's plan, universities researched basic science and then industry developed these findings to the point where they could get to market. The idea of R&D as two distinct activities was born. Firms soon organised themselves along similar lines, keeping white-coated scientists safely apart from scruffy engineers.

This approach was a stunning success. AT&T's Bell Labs (pictured above) earned six Nobel prizes for inventions such as the laser and the transistor. IBM picked up three, two from its Zurich Research Laboratory alone. And Xerox's Palo Alto Research Centre (PARC) devised the personal computer's distinctive elements, including the mouse, the graphical user interface and the Ethernet protocol for computer networking (although it was criticised for failing to commercialise such leaps forward).

Now the big corporate laboratories are either gone or a shadow of what they were. Companies tinker with today's products rather than pay researchers to think big thoughts. More often than not, firms hungry for innovation look to mergers and acquisitions with their peers, partnerships with universities and takeovers of venture-capital-backed start-ups. The traditional separation of research and development enshrined by Bush in 1945 is rapidly disappearing, especially in the information-technology industry. Does this mean the days when companies came up with big breakthroughs are over, too?

Not necessarily. The approach to R&D is changing because long-term research was a luxury only a monopoly could afford. In their heyday, the big firms dominated their markets. AT&T ran the telephone network, IBM dominated the mainframe-computer business and Xerox was a synonym for photocopying. The companies themselves saw the cost of basic scientific research as a small price to pay for such power.

Modern technology firms are much less vertically integrated. They use networks of outsourced suppliers and assemblers, which has led to the splintering of research divisions. Even though big American firms still spend billions of dollars on R&D, none has any intention of filling the shoes left empty by Bell Labs or Xerox PARC. The research and development that Bush tore asunder are once again becoming entwined. Old-fashioned R&D is losing its ampersand.

“The lesson learnt is that you don't isolate researchers,” says Eric Schmidt, the boss of Google, who started his career as a computer scientist at Bell Labs and later at Xerox PARC. The “smart people on the hill” method no longer works, he adds. Instead, researchers have become intellectual mercenaries for product teams: they are there to solve immediate needs.

This view is shared by other industry veterans. “The corporate research labs of the old days are really not going to be the basis of what is new,” says John Seely Brown, the director of Xerox PARC for over a decade until 2000. “This is getting to be a new kind of game.”

On the surface, American innovation has never been stronger. American firms spend around $200 billion on R&D annually, much of it on computing and communications. Microsoft, for example, spent around $6.6 billion last year; IBM and Intel about $6 billion each; and Cisco Systems and Hewlett-Packard (HP) around $4 billion each. Most of this money went into making small incremental improvements and getting new ideas to market fast.

For example, IBM has eight laboratories on three continents, each with its own personality and expertise. At its Zurich Research Laboratory around 300 scientists representing over 20 nationalities concentrate on areas such as microelectronics, nanotechnology and computer security. Only a few years ago researchers were judged on the basis of patents and papers, but today they roll up their shirtsleeves and work alongside the company's consultants, explains Douglas Dykeman, one of the laboratory's managers.

This reflects IBM's transition into “services science”. The services business is becoming commoditised, as hardware did before it, and IBM knows it must add intellectual property to its offerings. Putting its researchers on the case is one way to charge clients a premium.

“Some days I used to sit there thinking ‘does anybody really care?'” recalls Mr Dykeman about some past projects. Now the laboratory has to apply science to real and immediate concerns. Often the distinction between “R” and “D” is blurred. For instance, a recent research paper from the laboratory about the bonding of gold atoms smacks of pure science. But as semiconductors move to the atomic scale, answering this sort of question will solve production problems ten years out.

The new science

In Redmond, Washington, Microsoft Research houses 400 researchers; it boasts another 300 around the world. Nearly all of their budget is spent on commercially orientated projects. In a windowless office Steven Drucker, a researcher on media applications, is unable to get his laptop to work with the projector. So he explains the future of home entertainment with a video on the computer screen.

He envisages a world where people will take telephone calls from their television sets, interact with content online and receive relevant advertisements. But making this vision work technically is hard. Academics, he says, cannot do this, since they continually struggle for funds. This forces them into projects of just one or two years—even shorter than industry horizons. “It's insane,” says Mr Drucker. He reckons it means corporate research can look farther ahead, do bigger things and risk more money for a big payout.

“In the real world, it is not just a big ‘D' and a big ‘R'—it's a continuum,” explains Rick Rashid, who oversees Microsoft Research worldwide. The company, he says, does perform basic research. “But we also work with our product teams to move those technologies into our products.” Microsoft has a team of a dozen people whose sole responsibility is to handle technology transfer. Sometimes researchers move from the laboratories to work with product teams, as Mr Drucker recently did.

The distinction between development and research is intentionally blurred at HP, epitome of the research-driven organisation. At its base in Palo Alto, the offices of its founders, “Bill” and “Dave”, are preserved in the laboratories where they pioneered products and principles like open-plan cubicles for employees and “management by walking around”. Among the company's senior scientists is Bernardo Huberman, a physicist and former PARC researcher, whose work bridges computer science, economics and sociology.

Instead of looking at fundamental questions about the universe, Mr Huberman's prolific papers identify how “prediction markets” can be used to identify successful projects inside companies; how to price a unit of grid-computing (broadly the harnessing of the collective processing power of many computers) and how to vie best for internet users' attention. The old model of research, of “putting people in a bubble”, is over, he says. The most interesting research is now done “where technology touches people”.

Having researchers work more closely with customers pays off in other ways. For example, HP's work for DreamWorks Animation SKG, a film studio, required a highly sophisticated video-conferencing system so executives could regularly talk face-to-face without having to leap on an aeroplane. What the HP laboratories came up with was so successful that the company commercialised the system as a product, called Halo. This is now used by other companies, including PepsiCo and AMD, a chipmaker.

Such schemes represent the primacy of “D” over “R”. “Is that a bad thing?” retorts Shane Robison, who oversees HP's technology strategy, with every pore visible on his three-foot-long face over the Halo teleconferencing system. Decisions about investment research are made by betting where the industry is heading, he says. HP still does some basic research, adds Dick Lampman, who heads HP Laboratories, but when people “celebrate the other model, they lose sight of what it takes to take a good idea and make it into an exciting product.”

The fusion of research and development is meant to solve the central shortcoming of Bush's plan: how to turn ideas into commercial innovations. Great ideas may moulder without a way to develop them. In America the link between industry and government-funded research was reinforced in the 1980 Bayh-Dole Act. This expected recipients of federal funding to patent their innovations as an incentive for them to leave the laboratory. So, for example, when Google listed its shares in 2004, Stanford University received around $200m worth since research by Google's co-founders on search algorithms had been partly financed by the National Science Foundation. Moreover, the rise of venture capital has smoothed the progress of new ideas into products.

But inside large companies the transfer has been so difficult that it provided a huge incentive to join the two churches of “R” and “D” together. “The idea devalues itself over time if you don't get it to market quickly,” says Paul Horn, who oversees IBM's research. “Everything we do is aimed at avoiding a ‘handoff'—there is no ‘technology transfer'. It is a bad phrase at IBM.” Research teams stay with their ideas all the way through to manufacturing.

The epitome of this practical emphasis may be Intel. The chipmaker relies on universities for much of its basic science and takes a highly disciplined approach when selecting and overseeing the projects it backs. Staff scientists are intimately familiar with manufacturing. Many of its patents cover making chips, not basic science. Research is better “the closer the development is to the brutal market reality,” says Sean Maloney, an Intel executive. “Our people have that tattooed on their tonsils.”

One reason for the shift towards more commercially minded research in technology companies is that the nature of IT has changed so much. In Bush's time the science that went into computing was itself closer to basic research. By contrast, many of the big scientific questions in computing have been answered—at least well enough for companies to find that innovation emerges from new ways of arranging today's technologies rather than inventing new ones. Dell's innovation was a business model that used extreme supply-chain efficiency to create bespoke computers. Likewise, Apple's iPod is a new interface atop standard industry parts.

Bush's approach also presumed that research was costly, if only because computing machines back then were expensive and huge. Moreover, he presumed that cycle times for development were long.

Researchers as developers

Today, innovation comes more from software than hardware, computing costs are extremely low and development times are very fast—almost immediate, in fact, when the internet is used to deliver them. That explains why the boundary between research and development is blurring most for web-based companies, where competition is fierce and the time to market can be instantaneous.

The Google method means researchers are part of development, says Mr Schmidt. The company employs very small teams to work on a small number of ideas, some of which may turn into big hits. Failure is an essential part of the process. “The way you say this is: ‘Please fail very quickly—so that you can try again',” says Mr Schmidt. Its headquarters, the “Googleplex”, is an environment for employees to wander around intellectually. Google has many projects competing for users' attention. By being a free service, and using the internet to reach customers immediately at low cost, online firms like Google can experiment in ways that were once impossible.

Meanwhile at Yahoo!, researchers are also close to the practical needs of the business. At the opening of the company's new European research centre in Barcelona in October, the laboratory took up offices just across the hall from the business and marketing divisions. Yahoo!'s corporate research unit, like others, has hit upon a strategy to tap into fresh ideas and young talent at universities. The laboratory is also located in the same building as the Barcelona Media Innovation Centre, which is affiliated with the nearby Universitat Pompeu Fabra.

According to Mr Brown, the former PARC director and now a senior fellow at the Annenberg Centre for Communication at the University of Southern California, internet companies benefit from a “combinatorial ideation platform”. By this, he means that ideas can be tried out fast, directly on consumers, who can also combine them with ideas of their own to do new, unanticipated things—and all with immediate market feedback.

The post-war R&D model needs updating not simply because of the new way of innovating, but also because companies now have a greater choice of where to shop for ideas. “If you go back to that period of AT&T, there wasn't the same kind of engineering going on at universities, and Bell Labs could not rely on an external cadre of engineers for their research—they really had to do it internally,” says Charles Giancarlo of Cisco.

He admits his company has long been accused of “R&D by M&A”—but he does not see why that should be a problem. Cisco spends $4.3 billion a year on employing over 16,000 engineers around the world, in addition to tapping into the venture-capital industry's start-ups. Basic research inside companies is impossible in a competitive industry, according to Mr Giancarlo. “We might decry this on a public-policy basis, but at least as far as public markets are concerned it is a Darwinian world. You live or die by that.”

Industry's abandonment of the split between R&D comes as computer scientists complain about a decline in basic research or its distortion by commercial influence. The National Science Foundation and DARPA (the Pentagon's research arm), two of the grant-giving institutions inspired directly by Bush's vision, have been chided for turning away from basic research in favour of later-stage work, with an emphasis on homeland security—just as industry has done. At the same time, America's venerated national laboratories are coming under partial industry control.

Some in industry regret the melding of R&D into one activity and the demise of the big corporate laboratory. At IBM, some researchers are not impressed by “services science”. And although Google's engineers can devote 20% of their work-time to their own projects provided it helps the company, the footsoldiers roll their eyes and admit that such time is usually found on Sunday afternoons.

The new model of R&D turns researchers into the shock-troops of innovation. Bell Labs, now a part of France's Alcatel-Lucent, is turning its attention almost entirely towards development. It reached a low point in 2003 when it had only some 1,000 researchers and a $115m budget. Half a century ago it claimed 25,000 researchers and its own physics laboratory. In 2002 Xerox PARC became an independent subsidiary able to provide research services and intellectual property to outside clients, not just its parent.

Some private institutions have sought to enter the field of “basic science with a business plan”, so to speak. But their success remains to be seen. In the 1990s Paul Allen, one of Microsoft's founders, set up Interval Research to great fanfare, but closed it within a decade after spending $100m with little to show. More recently, Nathan Myhrvold, Microsoft's former chief technology officer, created Intellectual Ventures, to come up with futuristic technologies and license them widely. It has filed over 800 patent applications, but has yet to publicise any big innovation.

Perhaps all this would have made Bush weep. “Industry is generally inhibited by preconceived goals, by its own clearly defined standards, and by the constant pressure of commercial necessity,” he wrote in 1945. “Basic research is performed without thought of practical ends.”

But the message from mammon is different. “When I started out running PARC, I thought 99% of the work was creating the innovation, and then throwing it over the transom for dumb marketers to figure out how to market it,” says Mr Brown. “And now I realise that there is at least as much creativity in finding ways to take the idea to market as coming up with the idea in the first place. I would have spent my time differently had I figured this out early on.”