Why wouldn’t you stick with a sure thing like Lehman Brothers?” John Zimmer, then a young analyst at the investment bank, remembers being asked when he said in 2008 that he planned to leave to work on a startup. It was, admittedly, a wild idea. Zimmer had happened upon his co-founder, Logan Green, on Facebook around a year earlier, and since then the two had been working part-time on Zimride, a startup trying to popularise long-distance carpooling. Success was a long shot. But Zimmer was 25 and undaunted. He moved from New York to Silicon Valley, driving a carpool across the country to raise awareness of Zimride along the way.

Within a few months, Zimmer’s former employer collapsed. Zimmer, meanwhile, was travelling around the country selling carpooling to university students, dressing up in a frog costume to attract attention. Zimride did not fail as dramatically as Lehman Brothers, but by 2012 it was running out of money. Hoping to devise another plan, Zimmer and Green held a “hackathon” in their office, located in a former auto-body shop in a seedy area of San Francisco, and came up with the idea of building a service where users could hail rides on their phones. Uber already existed, but it was exclusively for those looking to hire a professional driver in a swanky black car, not an average person driving his own. They named their new service Zimride Instant, and soon changed the name to Lyft. “Your friend with a car” was the founding tagline. Zimride’s earliest investor, Sean Aggarwal, thought the service sounded “nuts. A random stranger is going to show up driving his or her car, and you want me to jump in?”

What began as a lark has become a transformative feature of urban life. Around 30% of young Americans use ride-sharing services, and 15% of adults of all ages do, according to the Pew Research Centre. Lyft operates in 200 cities across America and is valued at $5.5 billion. It boasts high-profile investors, including the hedge-fund manager Carl Icahn.

Building such a valuable business so quickly is a spectacular achievement. But in Silicon Valley, the heartland of American capitalism, Lyft’s success is viewed only relatively. A few months after Lyft launched in San Francisco, its rival Uber started a similar service called UberX, and began spending billions of dollars to expand globally. Today Uber is the most valuable startup in history, valued at around $62.5 billion, around 11 times more than Lyft. “We’ve raised $2 billion. If Uber didn’t exist that would be the craziest thing ever,” insists Zimmer, with a hint of defensiveness.

But Lyft is stuck in Uber’s shadow. I asked one venture capitalist what he thought of Lyft. He put his thumb and index finger into an L shape, brought his hand to his forehead and mouthed “loser”. Uber’s boss, Travis Kalanick, is notoriously rude and pushy. Doubters’ primary criticism of Zimmer is that he is too “nice” to lead Lyft to success.

They are underrating him.

Zimmer drives calmly, patiently, as his passengers in the back seat direct him through Twin Peaks, with the downtown of San Francisco stretching out before us. Sometimes Zimmer volunteers as a Lyft driver to keep tabs on the service. On this Sunday in May, he never boasts to the unsuspecting passengers that he is the service’s co-founder. When two passengers find out, they ask to take a selfie with him before they pile out of his leased Volkswagen Jetta.

Lyft’s whimsical, playful brand reflects Zimmer’s personality. He is the firm’s public face; his co-founder Green oversees engineering and internal operations. From the start he suggested placing a pink, fuzzy moustache on the front of cars so customers could recognise who was picking them up. He wanted the service to feel fun and casual. “You’re used to sitting next to people, not being chauffeured in a Honda,” Zimmer explains. Customers were encouraged to sit in the front seat and greet their driver with a fist-bump when they got in.

Zimmer has a quiet humility and playfulness, and is not motivated by money. The most expensive thing he has ever bought for himself besides a laptop is a $1,000 bike, which he purchased after Lyft closed a recent funding round. The lack of materialism is fitting for someone whose firm has helped spread the “sharing economy”, where people rent rather than buy what they need. Zimmer’s motivating ambition is to reduce car ownership, and he likes to fantasise about how cities could be redesigned to give less space to parking and more to parks. His favourite books are an environmental textbook he read in college and “The Lorax” by Dr Seuss.

His Utopianism can sometimes make him sound like a Californian hippie, not someone who spent more than half his life in Greenwich, Connecticut, and used to work for Lehman Brothers’ commercial mortgage-backed securities division. Zimmer grew up surrounded by people striving for the trappings of wealth – some friends in Greenwich received BMW s for their 16th birthdays – but had a healthy resistance to the bling of his hometown and the fortune to be born to down-to-earth parents. “I am a bean counter,” deadpanned his mother, who is about to retire as an accountant for a coffee-bean importer, when I met his parents. She was wearing a silver necklace in the shape of a Lyft moustache and had misplaced her matching earrings.

As a teenager, Zimmer read about an Indian reservation in Pine Ridge, South Dakota, and asked his parents if he could volunteer there one summer. The pervasive poverty and close-knit community he observed there stirred him. He spent other summers interning at a nearby Hyatt hotel, where he learned the reward of keeping customers happy.

If there is a thread that runs through Zimmer’s life, it is exceptionally good timing. While a university student at Cornell’s School of Hotel Administration, Zimmer happened to take a class on green cities with an inspiring professor, named Robert Young. It was here that he became fascinated by the fact that cities are laid out to cater to cars, which go unused most of the time. Even when they are driven, 80% of seats typically stay empty. He started thinking about what would happen if the principles of hotel management were applied to automobiles. “From an occupancy perspective,” Zimmer explains, cars are “failing hotels”. While at university Zimmer spent a semester abroad studying Spanish in Seville, and needed directions to “Street of the Love of God” one day. He asked a passing local how to get there. They married in 2014.

Zimmer’s professional life has benefited from fortuitous timing too. He left Lehman only a few months before it failed. In 2012, when Zimride was metamorphosing into Lyft, smartphones had become ubiquitous, which enabled drivers and passengers to set precise locations for instantaneous pick-ups.

His stint at Lehman Brothers informed the sort of firm Zimmer would build. The investment bank’s culture, he says, was “broken”, and people put in miserably long hours just to prove they were working. Lyft’s offices are the anti-Lehman, with pink foam pool noodles hanging from the ceiling of the entrance hallway and statements like “create fearlessly” and “make it happen” painted on the walls. Lyft holds its board meetings behind a huge portrait of Willy Wonka, the zany confectionery magnate from “Charlie and the Chocolate Factory”, which swings open to reveal a hidden library. Employees look like fresh-faced university students, and seem genuinely happy to be at work. “Is this the business world? I think this is the community world. I don’t feel like I work for a corporation,” says one.

Zimmer is known for being mild-mannered and affable, but has had to become a strong negotiator. Within a few months of launching Lyft in San Francisco, a local-government body sent the startup a cease-and-desist letter and began what would become a cascade of regulatory resistance to ride-sharing. Ultimately Zimmer helped negotiate regulations that would allow Lyft to operate legally.

Golden ticket to ride John Zimmer with the Willy Wonka portrait leading to Lyft’s boardroom

For someone with a conscience, all the politicking can be exhausting. Taxi drivers around the country have opposed Lyft and Uber, because they charge customers less and take away business. When Lyft was preparing to open in Seattle, taxi drivers held up posters bearing a photograph of a young girl who had been hit and killed by an Uber driver. “You question yourself along the way,” he says. “You think about [taxi drivers’] lives, and you have to be sure that the net benefit of what we are doing is positive.”

This sort of reflective humanity is not in vogue, at least at Uber, where the company culture is cut-throat and competitive – more Lehman than Lyft. Travis Kalanick, Uber’s boss, is seven years older than Zimmer and his co-founder Green, and approaches expansion as one would wage a world war, deploying billions of dollars to win markets around the world. Uber’s main conference room is called the “War Room”. Even people who respect Kalanick think he is intensely rude, but believe that his aggression will help fight regulators and establish ride-sharing as mainstream.

Uber’s ascendancy over Lyft can be traced to Kalanick’s ambition and competitiveness. He is a more experienced entrepreneur than Zimmer, having launched several not-wildly-successful startups. One, called Scour, specialised in file-sharing and was sued over copyright infringement by several media companies seeking up to $250 billion in damages. It filed for bankruptcy in 2000.

From the beginning, Kalanick has been fiercely focused on ensuring Uber’s dominance. He has raised money as quickly as possible and pushed Uber to expand worldwide, so he could declare cities and countries “won” and intimidate rivals. Rather than working with regulators, he has favoured a more caustic approach, often choosing to operate illegally in cities and establish a foothold before reaching a compromise with regulators. He has described himself as fighting a political race “where the candidate is Uber and the incumbent is an asshole called ‘taxi’”. The low-key Zimmer has worked hard not to bait taxi companies, and has favoured a more measured, collaborative strategy for expansion. He has decided to focus on America, negotiating partnerships with local firms in China, South-East Asia and India, so Lyft’s users can get a car with them if they ever travel there.

In total, Uber has raised $12.5 billion – the most by any startup in history – from deep-pocketed investors around the world, including Saudi Arabia’s government. Lyft, in contrast, has raised $2 billion. Uber had $10.8 billion of gross bookings in 2015; Lyft claims around $1 billion (of which it keeps around 20%). Meanwhile, Kalanick has done everything in his power to propagate the narrative that only one firm – his – can win. When Lyft was seeking new investors a few years ago, Kalanick tried to scupper its fundraising efforts by calling people Lyft was talking to and trying to convince them to invest in Uber instead. A few years ago Uber employees reportedly hailed nearly 6,000 Lyft rides only to cancel them, hoping to upset Lyft’s drivers and cause them to defect to Uber. Colin Frolich, an early Lyft employee who now works at Airbnb, recalls how within a month of starting to work at Lyft, Uber blocked his account, effectively barring him from using the service.

The fierce rivalry between Uber and Lyft has prompted them to spend an unprecedented amount of cash – probably more than $1 billion – to lure passengers and drivers to their services in American cities where the two compete. But Uber has taken aim at Lyft in more public ways as well; a few years ago it ran an ad directed at Lyft’s drivers, encouraging them to “shave the stache” and join Uber instead. Lyft fired back a bit meekly, erecting a billboard that urged drivers to “be more than a number”.

Perceptions of success are always relative. People feel less wealthy around their richest neighbours, less intelligent around their smartest colleagues and less attractive around beautiful friends. But in the technology industry comparative assessments can be especially brutal. Like Hollywood, the tech industry is small and people closely scrutinise one another. A widely held view in Silicon Valley is that success is zero-sum. Being perceived as the winner in one’s sector is important for hiring talent, attracting investors and garnering the media’s attention, and the leader has a disproportionate advantage. “First prize is a Cadillac Eldorado. Second prize is a set of steak knives. Third prize is you’re fired,” Stewart Butterfield, the boss of Slack, a software startup, told me, quoting from David Mamet’s play “Glengarry Glen Ross”.

Silicon Valley’s obsession with being first has some legitimacy, because many internet businesses have proved to be winner-takes-all. Google, Facebook and Amazon are all companies that once had a lot of competition but came to dominate their respective spaces and eventually crushed rivals. Internet businesses thrive on “network effects”, which mean that they become more useful and powerful the more people sign up. For example, Google becomes better at predicting what people want as it performs more searches, prompting more people to use the service and giving it a disproportionate lead over competitors. The internet sector is different from most other industries, where competition is fierce, and being number two is not viewed as an insurmountable weakness. Think of Coca-Cola versus Pepsi, Nike versus Adidas, McDonald’s versus Burger King, and General Motors versus Ford.

Many people in Silicon Valley are biased against Lyft because they think that ride-sharing will be a winner-takes-all market. They are probably wrong. Once a ride-sharing company gets to a certain level of reliability and waiting times are short enough, there is little benefit to using one service over another. Uber and Lyft will both be able to claim a lead in lots of cities, and both will be around for a while.

Silicon Valley may have a reputation for being liberal and accepting, but it is an extremely competitive place that values ferocity. Aspiring startup founders see Steve Jobs, who was notoriously difficult to get along with, as an icon. “People tend to judge others by their success and not by their humanity,” says Randy Komisar, of Kleiner Perkins Caufield & Byers, a venture-capital firm that backs startups. He believes that “in hyper-competitive Silicon Valley, the winners are more likely to be self-absorbed personalities, where it’s all about them, their mission, their goals. It’s not about others.” In this respect, Silicon Valley has become like Wall Street and other high-intensity sectors, where the aggressive and cut-throat are expected to claim victory.

Zimmer’s view is the opposite. He is confident that nice guys eventually win, and points to Howard Schultz, the boss of Starbucks, as proof that wholesome people can be successful. “I’m competitive, but I want to treat people well,” he explains. “I think that’s an advantage. Our business is a service business.” Lyft must cater to both users and drivers, and some drivers do say they prefer Lyft’s more personable, humane corporate culture. He is not fazed by coming in second, and views the criticism that Lyft is too “soft” as a myth propagated by Uber. Uber’s aggression will cause it problems. Governments around the world, from France to India, have sought to ban the service. And Uber’s desire to conquer the Chinese market has led it to fight a costly war with a local rival; this could turn into an expensive distraction.

If, as Zimmer believes, cars will be self-driving in future, people are less likely to own them and more likely to summon them from Lyft-style companies. That might put Lyft at odds with the automakers; but instead of going to war with them, Zimmer has tried to win one over. He convinced General Motors to invest a staggering $500m in Lyft, and negotiated a partnership with them. He predicts (somewhat optimistically) that people will be able to hail self-driving cars through Lyft in at least some cities within three years.

Zimmer still has plenty of work ahead to help Lyft grow. The pink moustache might fit in well in zany San Francisco, but could limit Lyft’s appeal elsewhere. Zimmer realises this, and sent out an email to customers telling them that they no longer have to sit in the front seat and fist-bump their drivers. Lyft has taken its moustaches off car grills and put them on dashboards, and may yet try to make them more diminutive. Zimmer thinks that in the beginning Lyft had to run a primary election to win users, and has more recently turned to running a general campaign and becoming “centrist” to widen adoption.

Lyft’s smaller size could prove to be a virtue. Even though it’s worth billions of dollars, it’s still small enough to be bought by another company, securing a fortune for Zimmer and Green and offering them a chance to embark on another entrepreneurial venture. I ask Zimmer what he would do if he were to do another startup. It is something he has discussed at times with Green, he admits. They have talked about building a city and designing it differently, so the car would not dominate the landscape. It’s a zany idea. It could never work. But Zimmer, of course, has heard that before.