Illustration by Sonia Roy/Colagene.com

The cure for banks? Ed Miliband advocates more competition. Need to improve education? Nick Clegg urges more competition between students, between teachers, between schools. The solution to fuel poverty? David Cameron places his faith in competition between the energy companies. From TV talent contests and school rankings to the Olympics and rich lists, our religious faith in competition has promised fabulous efficiencies, miraculous economies and dazzling innovation. Instead we find ourselves gasping for air in a sea of corruption, dysfunction, environmental degradation, waste and inequality. Might there be a connection?

When I interviewed bankers for my 2011 book Wilful Blindness, the brutal pragmatism of a competitive industry was spelled out. “Sub-prime was about ripping off poor people,” one told me. “But we have to employ a sales force. And there was no way we could hire, let alone retain a single good salesperson without letting them sell these high-commission products.

“What were we supposed to do? Sit on our principles and watch as every salesperson walked out the door?”

The financial crisis proved so catastrophic because so many were selling the same toxic products. Classic economic theory may argue that competition is productive because it generates a diverse range of goods and services that benefit consumers and, by extension, society – but in this instance (and many others) it signally failed to do so. Belief in the theory underpins Cameron’s and Miliband’s touching faith that competition, and being more easily able to switch between banks or energy providers, will somehow liberate consumers from price-gouging. In fact, it seems more likely that it will just encourage companies to copy each other’s dodgy innovations at a faster rate. Competition frequently fails to deliver its theoretical promises. Intense competition inside and between institutions generates dysfunction, corruption, waste and the unwinding of the social fabric.

In organisations, competition for permanent jobs, bonuses and promotions can erode trust. Many companies formalise this through forced ranking, a system in which employees are assessed and rewarded for their position within a standard distribution. The top 10 per cent are winners, the bottom 10 are losers and are encouraged to move out, and those in the middle are (at least temporarily) safe. At Enron, this was known as “rank and yank”, at Intel “Focal” and at Microsoft “stack ranking”. The system is a crude form of social Darwinism, inspired by the hope that a need to survive will promote great work. In fact, it has just the opposite effect: people sabotage each other, appearing to be courteous while keeping back just enough information so that colleague-competitors can’t excel. Pleasers and politicians thrive, gaming a system that no one takes responsibility for; if you’re a winner, the system works for you – and if you’re a loser, it’s not your problem. Microsoft recently announced that it was abandoning the system but most large corporations still use it, and then wonder at their inability to innovate.

Competition can’t deliver the creativity these managers need because it specifically disables collaboration. If I’m being judged in comparison with my peers, why would I help them? That these executives are the products of competitive education systems only exacerbates the problem: they bring with them a lifetime of being trained to compete for class rankings, prizes and places. In the United States, where class rankings are still common, parents advise their children not to help one another, on the grounds that doing so may jeopardise their winning the top spot. Here in the UK, primary school teachers observe “competitive friending”: parents’ attempts to ensure that their kids make the right friends to enable acceptance in the right social networks.

In both the UK and the US, the emphasis on competition and ranking encapsulates the same message: everyone is a rival. This does little to teach the subtle habits of collaboration but much to focus any child’s mind on results. If grades are all that matters at school does it matter how you get them? The past decade has brought an explosion in cheating, plagiarism and the use of drugs to enhance exam performance. At the Institute for Global Ethics, the late Rushworth Kidder estimated that, by the time they reached college, 75 per cent of students had cheated – which is why many universities now run students’ essays through Turnitin software to check that they haven’t been copied or stolen.

In the world of science, a well-honed competitive mindset has produced what many leading researchers are calling a crisis: a culture in which the open exchange of ideas, data and theories has all but stopped. Crick and Watson may have considered themselves to be in a race – but their success hinged on the shared insights, data and debates of colleagues. They would find today’s labs very different: in 1966, 50 per cent of scientists said they felt safe talking about their research, but by 1998 that number had fallen to just 14. Science is a necessarily accretive process but from Harvard and Washington to London and Berlin, ambitious scientists wanting to be superstars share with no one. Rivalry and the fear of being scooped stop them from pitching in.

Progress for a scientist is measured in publications, citations and research awards – and as the competition for both has increased, so have fraud, plagiarism and what scientists call “normal misbehaviour”: secrecy, sabotage, data slicing and culling. At the University of Washington, Ferric Fang has grown particularly concerned about the increasing numbers of scientific papers that have to be retracted because they are rushed into print too fast, with inaccurate, incomplete or fabricated data. The number of articles published in the past decade has increased just 44 per cent but retractions of scientific papers have increased tenfold – and most scientists believe this represents the tip of the iceberg.

The cost of this is inestimable; flawed papers lead researchers down dead ends and deflect others from promising avenues. The fraud of the prize-winning physicist Jan Hendrik Schön (who falsely claimed spectacular advances in the field of nanotechnology between 2000 and 2002) cost numerous scientists years of fruitless work and wasted resources.

****

The tournament that is modern science has produced what scientists call “the Matthew effect”, according to which the well-funded scientists (winners) get more funding and those with little (losers) get less. This might be a great way to run a game show but it is an especially poor way of promoting discoveries because picking winners is so difficult. The history of science is replete with cases of stunning breakthroughs made by the least likely of people, from the Augustinian monk Gregor Mendel to the “surfer pothead” Kary Mullis, whose invention of the polymerase chain reaction (PCR) transformed the science of genetics.

The costs of competition in business are sometimes obvious – fraud, corruption, sabotage – but many are more oblique. The measure of a company’s success (or the status of its CEO) is size, and the pursuit of growth is routinely pursued with high-risk strategies whose true cost may be apparent only years later. This is what the legal scholar Lynn Stout calls “fishing with dynamite”.

The quickest way to grow a company is through mergers and acquisitions, an old headline-grabbing favourite of high-profile CEOs even though research shows a failure rate of anywhere between 40 and 80 per cent. Under John Browne, BP grew fast by buying Amoco in 1998, Atlantic Richfield in 1999 and Burmah Castrol in 2000. Theoretically this should have generated economies of scale but it created debt, which ushered in an era of cost-cutting.

Similarly, the quest to make RBS the world’s biggest bank left it with the biggest loss in British corporate history and, in 2012, with a balance sheet the size of the UK’s economic output. Many working at RBS sensed that the acquisition of the Dutch bank ABN Amro in 2007 was driven by Fred Goodwin’s desire to pull off the biggest deal in banking. The quest for scale delivers not just huge risk, but also vast complexity. Supersizing companies always comes at a cost because competitive instincts don’t stop until they fail. To this day, RBS is in a tangle that people working there don’t believe they can fix.

Competition for market share is typically pursued by lowering prices. This race to the bottom might look great to consumers – dresses for £5, cashmere jumpers for £40 – but the costs have to go somewhere and usually they are pushed down to the most vulnerable. We may imagine this is a relatively new phenomenon but it isn’t. The Triangle Shirtwaist Factory fire in New York in 1911 (146 deaths) was echoed a century later by the collapse in 2013 of the Rana Plaza factory in Bangladesh (which made clothing for brands such as Matalan, Primark and Walmart), in which 1,129 died.

Globalisation didn’t invent the race to the bottom but firms such as Li & Fung accelerate it. Acting as a broker between low-wage factories and the companies that use them, Li & Fung’s “Little John Waynes” scour the world from Vietnam to Bangladesh to sub-Saharan Africa in search of ever cheaper labour. This is no small business – in 2012 Li & Fung earned $20bn – and, in theory, the brokers monitor working conditions. But its suppliers have had several disasters, including a factory fire that killed over a hundred workers.

Whether you’re making clothes, fast food or cheap books, competing purely on price drives down labour costs, producing a casualised workforce whose greater needs are either ignored or met by the state: a form of corporate subsidy that companies rarely acknowledge but happily accept.

The true costs exacted by a harshly competitive culture can be seen in the flood plains of North Carolina, the epicentre of the global meat industry. It isn’t just the ten million hogs (concentrating in just one state waste equivalent to that produced by the entire human population of Canada) which make this region remarkable. Rapid consolidation of family farms threw people off the land with nowhere to go. Industrialisation didn’t bring in money or create jobs but left the predominantly African-American families living off food stamps, stranded in a wasteland dotted with lagoons of animal excrement, afraid to protest the high levels of ammonia, hydrogen sulfide, acetic and butyric acids emanating from the facilities.

As large meat conglomerates moved into eastern Europe, a tradition of silence made consolidation easy: within ten years, 600,000 hog farms in Poland and 90 per cent of Romania’s independent farms had vanished. Horse meat is a sideshow, compared to the damage done to the social fabric of such places.

Economists may call these “perverse outcomes” but they are the predictable outcomes of competition. If we place our faith in it, we shouldn’t be surprised by such antisocial effects. After all, if my win is secured at the cost of your failure, what connects us? In a society that believes in winner-takes-all, how can competition fail to generate increasing levels of inequality?

Competition enlivens routine with drama, but when the stakes are high, so are the costs. The ubiquitous metaphor of our age – sport – demonstrates how destructive competition is, when it comes to playing for the big prizes and huge rewards that professional athletes now pursue. Travis Tygart, the head of the US Anti-Doping Agency, and the man famed for bringing down Lance Armstrong, has long agonised over the increasing rates of doping and corruption that characterise elite sport. His research showed him that although people still valued sport for the lessons of fair play, collaboration, integrity and discipline it could teach, in reality they believed that all that really mattered was winning. “In a climate in which corporate executives fabricate financial records, citizens evade taxes, professional athletes commit felonies . . . cheating and unethical behaviour appear to pay off,” Tygart’s research concluded. “Is our nation well served by a citizenry that learns to prize winning and extrinsic rewards at any cost as the values held most dear?”

It’s a recurring question. How can we create schools, companies and communities characterised less by competition and driven instead by an intrinsic passion for innovation, problem-solving and collaboration? Crowdsourcing companies – Kickstarter, Airbnb, SnapGoods, RelayRides, TechShop and many more – start from the premise that it is pooling, not hoarding resources, that creates opportunity. These businesses are typically celebrated for their technology, but their true daring resides in their reliance on the human desire to work together.

More conventional businesses such as W L Gore and Arup have proved successful and resilient because they focus intently on building social capital – trust, reciprocity and shared values – both within the company and with all the other businesses they work with. This isn’t marginal; it is central to everything they do. W L Gore is known for producing Gore-tex but should be more famous for the way it runs its business; you succeed at Gore because people want to work with you, not because you’ve bested them in a contest.

The structural engineers at Arup have been able to build some of the most challenging structures in the world – the Bird’s Nest stadium in Beijing and the ArcelorMittal Orbit – because the firm nurtures a work environment in which people eagerly share expertise and where hierarchy and status contests are of negligible importance. That these companies are also owned by their employees isn’t the single driver of collaboration but consistent with a mindset that sees shared respect and commitment as the necessary conditions for progress.

The Business Secretary, Vince Cable, and others have been keen to champion employee ownership structures as making a difference to the way companies behave. They are right to do so but wrong to think ownership alone will immunise companies against the ills that competitiveness spreads. We have seen the Co-op mired in scandal and fiasco because its ownership structure proved insufficient to ward off the conventional allure of mergers and acquisitions, the quest for scale for its own sake.

There is a lesson here for nations also. While presidents and prime ministers posture on the world stage, comparing their standing in GDP league tables, it is the smaller countries, such as Finland and Singapore, that prove most agile. They have to be great partners because they don’t have the size or market heft to protect them. They export more, plan further ahead and learn quickly. Knowing they can’t win through dominance, smaller countries have had to develop the capacity internally to be excellent collaborators externally. Not surprisingly, their high-achieving school systems seek success for every child, because they don’t believe they can afford to waste anyone.

Larger nations find it increasingly difficult to adjust to a world in which partnerships, alliances and trust represent the best social and political capital. Britain’s agonised relationship with the European Union demonstrates just how poorly we have developed the ability to contribute the best of our talents to the best of our partners.

If we are to find new ways to live and work together, we need to develop and prize high levels of trust and give-and-take: elements that competition so subtly corrodes. We need to celebrate the individuals and institutions that produce the greatest opportunities for the largest number of contributors. Many companies around the world continue to prove the human capacity for this way of working and measuring collective success.

Yet many politicians, wedded to gladiatorial combat and the rankings mania of opinion polls, have signally lost the capacity to think beyond the narrow confines of a very short race. Our politics are stalled because our problems are complex and our means of addressing them are often crude and rigid.

In the looming face-off between business, governments and society, a competitive mindset can frame the contest, but accepting this could destroy the mental maps that might show the way towards a solution. The problem is a failure not of the imagination, but of courage: the willingness to relinquish fantasies of winning in exchange for the bigger prize of joint achievement and shared progress. l

Margaret Heffernan is the author of “A Bigger Prize: Why Competition Isn’t Everything and How We Do Better” (Simon & Schuster, £14.99)