Vancouver is Hollywood’s urban body double. It is famously the stand-in for New York, LA, Seattle and Chicago, employed when those cities just get too tough, too traffic-clogged, too murderous or too bureaucratic to film in. It is almost never filmed as itself. That is because, lovely as it is, it is also, well ... a little dull. Who would want to watch a film set in Vancouver? To see its skyscrapers destroyed by aliens or tidal waves, its streets populated by cops and junkies, its public buildings hosting romantic reunions? Yet Vancouver (original name, Gastown) has also spent more than a decade at the very top of the charts of the best city to live in the world. Can that really be right?

No. Not at all. In fact, Vancouver’s boringly consistent topping of the polls underlines the fundamental fault that lies at the heart of the idea of measuring cities by their “liveability”. The most recent surveys, from Monocle magazine, Forbes, Mercer and The Economist, concur: Vancouver, Vienna, Zurich, Geneva, Copenhagen and Munich dominate the top. What, you might ask, no New York? No London? No LA or HK? None of the cities that people seem to actually want to emigrate to, to set up businesses in? To be in? None of the wealthiest, flashiest, fastest or most beautiful cities? Nope. Americans in particular seem to get wound up by the lack of US cities in the top tier. The one that does make it is Pittsburgh. Which winds them up even more.

The big cities it seems, the established megacities of the US, Europe and Asia are just too big, too dangerous, too inefficient. So what do these top cities have in common? How exactly do you measure “liveability”?

All the surveys use an index. But what is on it? “There’s always proximity to nature,” says Tyler Brûlé (editor of Monocle and patron saint of liveable cities and airport lounges, whose column appears weekly in the FT’s Life & Arts section). “Global connectivity is important, education and we’ve recently added chain store metrics – is there a Starbucks or a Zara?” he says.

So that’s the mountains, lakes and huge cups of generic coffee accounted for. Then there’s efficient public transport (that faint whoosh is the sound of London, NY and LA disappearing). There are also cultural institutions, global connectivity, green urban policies, well-designed housing within an easy commute, and so on. Each determinant on its own seems an indisputably good thing. But what do they mean together? Can Munich (Monocle’s Number 1) really be one of the best places in the world to live? On a Sunday afternoon?

To even begin to understand how these slightly unsettling results are arrived at, we need to understand who compiles them and who they are for. The lists are made by well-travelled academics, researchers and journalists for corporate, media and creative executives on generous expense accounts as well as other academics enjoying grants and stipends. And, of course, by Tyler Brûlé.

Most of these people are profoundly concerned with things like well-designed street furniture, a proliferation of eye-wateringly expensive artisanal retail, boutique hotels with good (English-speaking) service and environmentally friendly mayoral policies. Certainly these are all things which help but they skew the polls to a particular type of European or marginal Pacific city. What they also do is to strip out all the complexity, all the friction and buzz that make big cities what they are.

I spoke to Joel Kotkin, a professor of urban development, and asked him about these surveys. “I’ve been to Copenhagen,” (Monocle’s Number 2) he tells me “and it’s cute. But frankly, on the second day, I was wondering what to do.” So, if the results aren’t to his liking, what does he suggest? “We need to ask, what makes a city great? If your idea of a great city is restful, orderly, clean, then that’s fine. You can go live in a gated community. These kinds of cities are what is called ‘productive resorts’. Descartes, writing about 17th-century Amsterdam, said that a great city should be ‘an inventory of the possible’. I like that description.”

Joel Garreau, the US urban academic and author, agrees. “These lists are journalistic catnip. Fun to read and look at the pictures but I find the liveable cities lists intellectually on a par with People magazine’s ‘sexiest people’ lists.”

Ricky Burdett, who founded the London School of Economics’ Cities Programme, says: “These surveys always come up with a list where no one would want to live. One wants to live in places which are large and complex, where you don’t know everyone and you don’t always know what’s going to happen next. Cities are places of opportunity but also of conflict, but where you can find safety in a crowd.

“We also have to acknowledge that these cities that come top of the polls also don’t have any poor people,” he adds. And that, it seems to me, touches on the big issue. Richard G Wilkinson and Kate Pickett’s hugely influential book The Spirit Level: Why More Equal Societies Almost Always Do Better (2009) seems to present an obvious truth – that places where the differential in income between the wealthiest and the poorest is smallest tend to engender a sense of satisfaction and well-being. But while it may be socially desirable, that kind of comfort doesn’t necessarily make for vibrancy or dynamism. If everybody is where they want to be, no one is going anywhere.

“Sure, Vancouver is beautiful,” says Kotkin, “but it’s also unaffordable unless you’re on an expense account and your company is paying your rent.” Burdett agrees: “Economically all these cities at the top of the polls are also in the top league.” In fact, it can often be exactly the juxtaposition of wealth and relative poverty that makes a city vibrant, the collision between the two worlds. Where parts of big cities have declined, through the collapse of industries or the fears about immigration that led to what urbanists have termed the “donut effect” (in which white populations flee to the suburbs, leaving minorities in the centres), there is space to be filled by artists and architects, by poorer immigrants arriving with a drive to make money and by the proliferation of food outlets, studios and galleries. These, in turn, attract the wealthy back to the centre, at first to consume, and then to gentrify. Whether in New York’s SoHo, Chelsea or Brooklyn, in Berlin’s Mitte or London’s Shoreditch, Hoxton and now Peckham, it is at these moments of radical change that cities begin to show potential for real transformation of lives, or for the creation of new ideas, culture, cuisine and wealth. Once gentrification has occurred, bohemians may whinge about being priced out, as they always have done but, in a big enough city they are able to move on and find the next spot.

In a strange way the everyday conflict with the (unliveable) city can also become part of the attraction. Professor Tony Travers of the LSE says, “At one level the kind of urban sophisticates who live in these areas, in Hoxton or Brooklyn, want to fight the city. The urban struggle is part of the self-image of living on the edge.”

If the relative poverty of newcomers to the city distorts income equality in one direction, then the arrival of the super-wealthy does the same from the other end. The recent turmoil in the Middle East has led to a huge wave of investment in London property, one of the traditional safe havens for foreign money. London, unlike many cities that appear high on liveability lists, has few controls on property ownership. “If cities are any good,” says Travers, “they’ll attract a footloose international crowd who bring wealth.” And so the gap gets bigger.

He adds: “But they also come because of stability. If they buy something, they’ll be able to get their money back.” Which explains why New York and London remain popular, desirable and hugely expensive, despite never appearing on the lists.

The big cities also suffer from size. It’s true that Tokyo (Monocle’s Number 4) occasionally makes it on to these lists but metropolises like London, New York, Paris and Istanbul struggle with aged infrastructure and vast, sprawling transport systems. They are penalised in surveys for their inefficiency compared to, say, a small Scandinavian city. But it’s easy to be efficient when you’re small and when you have a highly taxed, wealthy population. It is also easy to initiate green measures, from recycling to cycling, which prove far more challenging in a proper metropolis with its problems of crumbling infrastructure and mobile population.

Yet it is proven again and again that the biggest cities are in fact the greenest. Their density, the close proximity in which people live and the minimal amount of land they occupy – compared with largely suburban Vancouver, for example, makes for a far smaller carbon footprint. Mumbai is probably the greenest big city there is – slums like the million-strong Dharavi use minimal land, energy and water. And, of course, without wishing to patronise, it is undeniable that there are happy people living surrounded by their families in Brazil’s favelas and millions living lives of drudgery and lonely despair beneath northern Europe’s leaden skies. The world’s most liveable informal cities lists have yet to be pioneered.

There is one criterion which throws up shockingly counter-intuitive results – beauty. On this criterion alone, almost any Tuscan hill town, perhaps Venice, perhaps Paris, would come out on top, yet none of these are there. Most of the beauty in the cities which occupy the tops of the leagues seem to ghettoise their beauty outside the city. They have convenient escapes, though the most beautiful and enjoyable – Rio, San Francisco and others – are curiously absent from the lists. The problem is that beauty doesn’t do you any good at all. It’s not a factor for the efficient, mid-sized chart toppers – though places such as Zurich certainly have their lovely bits. But it also damages your chances of making it into the disaffected megacities mentioned at the start of this article. The most beautiful cities become monuments to their own elegance, immobile and unchangeable. They cannot accommodate the kind of dynamic change and churn that keeps cities alive. In London, New York and Berlin, it is their very ugliness which keeps them flexible.

“The other big question,” says Kotkin, “is can someone coming from somewhere else improve themselves, reinvent themselves? Is there upward mobility?” The top cities score badly again. London and New York are magnets for immigrants precisely because they allow those kinds of new beginnings. They do have class structures but they are increasingly malleable.

There is one problem, though, that remains hard to ignore – violence. Johannesburg may be beautiful but its per capita homicide rates are astronomical; Los Angeles and New York are held back for the same reason. Washington DC’s per capita homicide rate, for example, is more than 30 times that of London and this continues to hold US cities down in the rankings. Urban guru Richard Florida remarks that the key to liveability is to “ensure that a city can guarantee the safety of all its residents”.

Of course, the ultimate difficulty with these surveys is that tastes are individual. I find London infuriating but –with the possible exception of New York – couldn’t think of anywhere else I’d rather live. “The city is a unique and private reality,” wrote Jonathan Raban, author of Soft City. He proposed that his London was a “soft city”, a place that everyone remakes in their own manner, in which every place evokes a personal memory or connection and which we navigate through our own unique mental maps. Our cities are our own – we make them inside us. No city means the same to two people so how on earth can we measure them?

Edwin Heathcote is the FT’s architecture critic

..................................................

Rankings: the best and the worst

New York

The only city that gives me a thrill every single time I walk through it. Fast, furious, brash, cosmopolitan yet completely self-absorbed, it is the perfect big city.

Rio de Janeiro

An extraordinary cocktail of pleasure and pain, beaches and favelas, condos and shacks, Rio is one of the most beautiful, most tolerant and most varied cities on earth. Unfortunately, its high murder rate discounts it from traditional best cities lists. But what a cityscape.

Istanbul

The fulcrum of the delicate balance between Europe and Asia, Christianity, secularism and Islam, Istanbul manages to be both one of the most beautiful cities on earth and yet accommodating to huge and constant change. It is a young, international, wildly commercial city with an extraordinarily vibrant street scene, open 24 hours and genuinely alive.

London

London seems to have the ability to reinvent itself. It has been a magnet for immigrants for centuries and remains a place where the poor can make something of themselves and the wealthy can enjoy their money. Its infrastructure is crumbling, its property overpriced and its weather dull but London’s cultural life is astonishing and most of its museums are free.

Rome

It might be more than 1,500 years since Rome was a proper world city but its allure lies in a blend of history, chaos, beauty and infinite layers of culture.

A few that don’t make the grade

Moscow

Impossible to traverse on foot, infinitely rude, corrupt, understandably alcoholic and seriously traffic-clogged, Moscow needs work. It does have some beautiful bits, from the Kremlin through to the masterworks of revolutionary modernism but the legacy of the communist police state hangs heavy.

Dubai

Everything that could go wrong with a city does here. It is, in fact, a place with no “here”. A succession of malls, highways, hotels and hideous towers, it has spent its history announcing its arrival but hasn’t a clue what to do when it gets there.

Birmingham

Once it was the workshop of the world, an astonishing morass of industry that somehow threw up a powerful, elegant Victorian city, which has been completely destroyed. Its decline has been less complete than that of, say, Detroit or Flint but it manages to be uglier nevertheless.

Jerusalem

I know, I know – beautiful, holy, history lingers in its every shady corner. Yet the treatment of Arabs as second-class citizens, the ghastly security wall smashing through its edges and the omnipresent guns have spoilt it. Jerusalem is the perfect example of why tolerance is so critical to a city.