As the proverb goes, it takes a village – not just to raise a child, but to keep the village alive. We need planners and engineers to work together to maintain the infrastructure of the village, parents and teachers to jointly care for our children, and juries and judges to decide our court cases. Working together is an unavoidable part of modern life and, as we develop more tools to communicate with others across the world, tasks once delegated to individuals are now the work of teams.

Take science, for instance; in the past several decades, there’s been a huge shift towards teamwork. Whereas academics used to publish scholarly papers on their own, that work is now dominated by teams. It seems self-evident that a group of experts could produce more insights than a single scientist, but a team faces a different set of challenges than an individual. Only in recent years have we begun to hone in on how to shape a smart team, not just a collection of smart individuals.

A strong team can work faster and better than a single smart individual – but not all teams are strong. Managing a smart individual is hard work, but managing an effective team requires extra attention. In addition to addressing individual needs, a team must work together – a goal best accomplished in an environment that maximises each individual’s positive contributions while downplaying his shortcomings. Over the past decade, study after study has attempted to decipher and bottle the qualities of the ‘smart group’. Just as psychologists have tried to uncover the ‘g’ factor responsible for an individual’s general intelligence, they’re digging into the ‘c’ factor – the secret sauce of collective intelligence. And most importantly, we want to know how to bring that ‘c’ factor to all our collaborative work, whether that’s in the boardroom, the classroom, the lab, backstage, the woods or even in space.

Humans have been working together long before we took an interest in finding out how to perfect our methods. We’ve always hunted and scavenged together, made fire together, built villages together. There is power in numbers – not only safety, but also aggregate wisdom. One of the first formal observations of this came from Sir Francis Galton while at the 1906 West of England Fat Stock and Poultry Exhibition. It had a contest to guess an ox’s weight; entrants paid sixpence to submit their guess, and a prize was promised to the closest guesser. While none of the 787 entrants of the contest guessed the ox’s weight correctly (1,198 lb), the average of their guesses was fairly spot-on: 1,197 lb. ‘This result is, I think, more creditable to the trustworthiness of a democratic judgment than might have been expected,’ wrote Galton in a letter to Nature.

We tend to disdain averageness – we like to think of ourselves as above average, even if it’s impossible for all of us to be – but averageness can represent the best of humans. When you average a bunch of our faces, the collective us is more attractive than any one individual; if you average a bunch of our guesses, the collective us is more intelligent.

Scientists have leveraged the power of the crowd in citizen science projects such as NASA’s Clickworkers, where science enthusiasts looked at photos of the Moon to identify crater features. The average person’s contributions were as good as experts’ – and they managed to provide far more ratings than any group of experts could manage. The project was so successful that NASA has started a new site using the same technique, where students along with the interested public help map Mars by looking at photos taken by satellites and rovers, and tagging features such as craters, land and sky.

simulations of proto-language suggest that we happened upon language through an iterative process that resembles other species’ swarm intelligence

Large groups of trained amateurs can produce even better outcomes than experts. Through the Good Judgment Project, led by a team of decision-makers and economic experts, and funded by the United States’ Intelligence Advanced Research Projects Activity (IARPA), thousands of volunteer forecasters were asked to predict national security issues and world events, such as the probability of terrorist attacks or clashes between nations. These volunteers had no particular expertise, besides being briefed on forecasting pitfalls to avoid, but collective predictions outperformed security experts’ by about 30 per cent.

In some cases, collective intelligence emerges from no order whatsoever. In non-human creatures such as fish, bees, ants and even bacteria, individuals form ‘swarms’ to coordinate complicated behaviours such as group size and where to forage and build homes. Through swarming, humans have created things such as Wikipedia, which has no central leadership but produces reasonably accurate encyclopaedia entries. Human language might be the result of swarming; robotic simulations of proto-language suggest that we happened upon language through an iterative process that resembles other species’ swarm intelligence.

Now there are several new projects seeking to harness humans’ collective intelligence through swarms. One is Unanimous AI, or UNU, a platform that uses crowd opinions to predict future events. So far this year, it’s hosted swarms that successfully predicted the superfecta – the first, second, third and fourth places – at the 2016 Kentucky Derby (which led to quite a handsome payout for those betting with UNU’s predictions), as well as 11 of 15 Oscar winners in 2015.

UNU is organised into themed rooms where any user can ask any question she likes. Questions appear on the screen above a hexagon; each point of the hexagon represents a possible answer. In the middle of the hexagon is a puck that represents the people’s will; if the hexagon were the walls of a cell, the puck would be its nucleus. Each person in the group controls a digital magnet she can use to drag the puck towards the answer of her choice. The whole set-up reminds me of childhood sleepovers, where we’d all huddle over a Ouija board; as we asked the ambient ghosts how they’d died, our collective consciousness guided the planchette around the board, spelling out a spooky word, or nonsense – or our deepest fears and dreams.

UNU is more complex than a Ouija board – it uses an algorithm to manage chatters’ responses. For instance, the closer a user places her digital magnet to the puck, the more ‘pull’ she exhibits. Essentially, this manner of voting allows UNU to aggregate collective behaviour as though we were an animal swarm, but with built-in integrity to guard against the usual threats to integrity in a human group. The model provides anonymity. And the real-time aspect of the system helps to overcome human biases, such as voting for what everyone else is choosing.

It could be just a coincidence, but there’s converging evidence that the swarm is pretty good at its predictions – perhaps even better than just the average crowd’s. In 2015, NPR recreated Galton’s original ox-weighing experiment by polling more than 17,000 people, whose average guess came pretty close: 1,287 lb for a cow that actually weighed 1,355 lb. UNU, on the other hand, recruited a swarm of only 49 people, but their average guess – 1,250 lb – was also pretty close. UNU’s founder Louis Rosenberg has observed that letting people swarm leads to more effective answers; if you take those same 49 people and ask them to guess individually, their average guess is markedly worse (1,137 lb).

Even if swarming or other ‘wisdom of the crowd’ measures provide excellent results, the logistics of this type of crowdsourcing doesn’t lend itself very well to the problems that real teams face. It would be impossible to crowdsource a complex project such as conducting an experimental drug trial or sending a human to the Moon, projects that require big-picture visions but also the divvying up and completion of smaller tasks. In these cases, specialised teams need to discuss and collaborate to meet specific goals. So how do we build a smart team?

Collaboration is the hard part, and this is where teams can fall apart. Individuals come to a team with a whole host of cognitive biases, and while one’s intuition might be that a diversity of perspectives could mitigate those biases, collaboration can actually amplify biases such as our tendencies to overestimate how much control we have over events and how much we can generalise from a small sample of data.

Group work also discourages us from making mistakes. While this might sound like an overall good thing, consider that failure is an important part of learning. Acknowledging failure is the hard part of this learning process. But in a group, admitting failure can be a hit to your ego and reputation – an obvious disincentive to owning and learning from your mistakes. In teamwork, we can fall prey to our basest of human desires – being loved, respected, and seen as competent – and avoid owning up to our mistakes.

Plus, when mistakes are made in a team, it’s more difficult to identify the origin of failure as members become complacent in their belief that their teammates have made the right decisions. Imagine a nurse who discovers that a patient has been on an IV drip containing the wrong medicine for hours after their surgery. The team of experts caring for this patient is vast. It’s difficult to pinpoint where the process went wrong, and how it could be remedied. Did another nurse get two IV bags mixed up? Or was it the pharmacy that mistakenly sent the wrong medicine? What about the other specialists who saw this patient but didn’t inspect carefully enough to catch the error – are they responsible too?

the real danger of groups is not authoritarian rule, but the type of quiet complacency that morphs into overconfidence

A key factor in these types of mistake is complacency, a hallmark of group behaviour. To preserve unity, each individual member avoids being ‘the difficult one’ who rocks the boat; doubts go unvoiced. Insularity can exacerbate the problem; a team might descend further into its own cocoon, writing off any indications that its decisions or plans won’t work, and distancing itself from potential naysayers by viewing outsiders as dumb or even malicious. Complacency in a team’s expertise becomes so entrenched that it turns into overconfidence.

The psychologist Irving Janis was the first to study this phenomenon, which he dubbed doublethink as a nod to George Orwell’s 1984. Using historical examples of political and military disasters such as Japan’s attack on Pearl Harbor as case studies, Janis posits that the real danger of groups is not authoritarian rule, but the type of quiet complacency that morphs into overconfidence. Not all groups fall prey to groupthink. But if a group does, it can be hard to see from the inside.

For instance, Janis’s research indicates that though the US military had been warned about the potential for an attack at Pearl Harbor, they were overly confident in their safety. They rationalised this complacency by telling themselves that the Japanese would never attack, since it would inspire a full-fledged war. This complacency took them right up to the attack; when Admiral Husband E Kimmel, then commander-in-chief of the US Pacific Fleet, was told that contact had been lost with Japanese aircraft carriers, he reportedly joked: ‘What, you don’t know where the carriers are? Do you mean to say that they could be rounding Diamond Head and you wouldn’t know it?’ Unfortunately, they were.

So how do we avoid these pitfalls? To combat groupthink, it helps for teams to get out of their usual rhythm. One way to do this would be to break up into smaller groups that can produce a more diverse set of ideas, providing more perspectives for the team at large. Inviting outside experts to share their views at meetings can also shake up the team dynamic, and prevent members from becoming too complacent about their own ideas. Having a designated avenue for people to voice minority or dissenting views, or cultivating a work culture where people feel safe giving their honest opinion, can add much-needed alternative perspectives to the discussion.

Aside from combatting groupthink, there are other design features that can increase a team’s likelihood of success. One is to examine the size of the team. Based on research at 15 large, multi-national companies such as Nokia, the BBC and Reuters, teams of more than 20 have trouble coordinating; there are just too many people to manage and keep track of. Large, diverse teams of specialised experts are less likely to share knowledge and resources with one another, and are less likely to support one another’s workload – for instance, shifting around roles to accommodate individual needs.

Who is in the group is also key. An effective group is not just a matter of picking strong individuals. Rather, it’s a blend of individual strengths, weaknesses, biases, work styles and preferences, which interact differently. Add baking soda to sugar, eggs and flour, and you get cake batter, but add it to dish soap and vinegar and you get a bubbly, foamy mess; people, too, mix differently with others.

What you don’t want is an overzealous hotshot. While power can boost performance in tasks when an individual is working alone, nothing destroys team dynamics faster than a leader with an ego. When people are tasked with leading a group, that power can go to his head; you can see this in your micromanaging boss, your overzealous rec basketball captain, your meddling PTA board president. The powerful are often less polite, less empathetic, and more concerned with maintaining their own reputation and authority than with the success of the team.

So it follows that you certainly don’t want a team of hotshots. While it might sound like a good idea to call on the best of the best, doing so can lead to team strife. Egos will be tested and that can distract from the task at hand. It turns out that there’s a delicate balance of big egos and subordinates necessary for a productive group. On NBA and FIFA World Cup teams, performance peaks when roughly half the team members are all-star athletes. Any more than that tips the balance, and performance drops. This drop in efficacy also occurs with teams of high-power Wall Street analysts; productivity peaks when groups are around 50 per cent all-stars. In this sense, we are no better than chickens, who show a similar pettiness: if too many dominant, high-egg-producing chickens are put in the same colony, cumulative egg-production decreases.

This even holds true when you create pretend all-stars. When researchers randomly assigned one person in a group to have power over subordinates, they found that even a little bit of imaginary authority went to people’s heads; judges who rated groups on their mock business plans observed that the fake powerful become more concerned with their status within the group, and less focused on the task. This behaviour affected the rest of their group, too; groups with these appointed all-stars became less likely to reach agreement.

To avoid this power clash, teams can try to define roles and tasks before beginning a project; this prevents team members from devoting their energies to jockeying for position. But most importantly, team members must be committed to leaving their big egos at the door. Researchers have identified social sensitivity as an integral part of the ‘c factor’, a group’s intelligence. What follows is what good leaders already know: create an environment where people feel that their contributions are valued, and that group will work better together. Researchers have found that groups in which all members talk about the same amount do well. No one benefits when the boss dominates the discussion; team members want to feel heard, and they will be more motivated to share ideas and offer honest feedback when they feel like their contributions are valued.

An office’s worth of subtle, rarely acknowledged emotional labour typically falls to women

Part of working well together stems from knowing your teammates well enough to read their subtle cues: Rob rubs his eyes when he’s bored; Kim misses a beat in response when she’s thinking of a polite way to say it’s a bad idea. This also means learning how to read teammates well enough that you can appreciate them in the ways they want to be appreciated, and productively resolve conflicts.

Given how social skills play into a group’s ‘c’ factor, perhaps it’s not surprising that researchers have also found that groups with women tend to outperform male-heavy groups. Politeness and empathy are among the many social skills women are expected to master, like how to direct criticism with sensitivity, and how to diffuse office politics with smiles and small-talk. An office’s worth of subtle, rarely acknowledged emotional labour typically falls to women; if you work among peers, ask yourself: who on your team brings in baked goods for your colleagues’ birthdays? Who picks up after those office parties? More often than not, those are the people picking up on every tiny cue at your weekly all-hands meeting.

High-functioning teams also keep lines of communication open, and create designated times and spaces to foster personal relationships. This is why so many new companies are designing campuses with social areas built in: organic, impromptu conversations at the gym or in the cafeteria can build relationships and, perhaps, foster new ideas. As more teams go digital, springing for periodic in-person meetings or retreats can boost team productivity.

Of course, there’s no proven formula for building meaningful relationships. One avenue is brutal, unflinching honesty, as in the case of a Google manager who jumped into intimacy with his team by telling them he had stage-4 cancer. From that point on, his team shared personal details with one another, which opened them up to more honesty about what was and wasn’t working for them in the office – and led them to be one of the most productive teams at Google.

Another option for building team rapport is humour – good-natured humour, that is. Jokes in poor taste can make team members feel alienated. But in a best-case scenario, humour helps teams stay interested in their work, which can improve morale, especially as more employees expect work to be fun. Humour also makes team members feel closer to one another, even in work hierarchies, and can foster the trust and honesty that helps teams get things done. During tense disagreements or debates, a well-placed joke can lighten the mood and get a team back in sync with one another.

Avoiding groupthink, eliminating ego-driven behaviour, and fostering trust and openness can give teams a better shot at being successful, but implementing all these ideas is a real challenge. And it’s worth noting that what constitutes a team’s success is a moving target. Each study illuminating evidence for what makes a ‘good team’ is just a snapshot of how a specific group of people performed in a specific task in a specific situation. Real life is much messier, and largely out of our control. Still, like teamwork itself, the potential payoffs of cracking the ‘c’ factor are huge – and we’ll keep striving together until we do.