Technically, Robert Caro’s book The Power Broker is a biography of urban planner Robert Moses, but that description feels laughably inadequate on multiple counts. For more than four decades, this particular urban planner was the most powerful man in New York, an unelected emperor who dominated the mayors and governors who were supposedly in charge, and who physically reshaped the city through sheer force of will. Caro’s enormous book, meanwhile, is less a life story than an epic, meticulously detailed study of power in general: how it’s acquired, how it’s used to change history, how it ultimately corrupts those who get it.

First published in 1974 – Barack Obama read it aged 22, and was “mesmerised” – The Power Broker was released in the UK for the first time this year. But its themes are too timeless to seem dated. Like the multivolume biography of Lyndon Johnson for which Caro is best known, you might call The Power Broker “unputdownable” – except that, at 1,300 pages, putting it down occasionally is the only way to avoid sore muscles.

You needn’t care especially about New York to be awed by the changes Moses wrought there: during a 44-year reign, he built nearly 700 miles of road, including the giant highways that snake out of the city into Long Island and upstate New York; 20,000 acres of parkland and public beaches, plus 658 playgrounds; seven new bridges; the UN headquarters, the Central Park zoo and the Lincoln Center arts complex, racking up expenditures of $27bn, dwarfing any previous run of construction in US history. “In the 20th century,” wrote Lewis Mumford, “the influence of Robert Moses on the cities of America was greater than that of any other person.” Around 500,000 people, who happened to find themselves in the way of Moses’s vision, were evicted from their homes. Did he drag New York into the modern age, forcing through much-needed public works and eradicating intolerable slums, against opposition from corrupt politicians and landowners? Or did he nearly destroy the city, subjugating its human inhabitants to the sovereignty of the car?

Caro, a former newspaper reporter, doesn’t pretend to be neutral: note the book’s subtitle. In Caro’s telling, Moses started out an idealist, inspired by his mother, a pillar of the New York German-Jewish community, whose zeal for helping the less fortunate was matched by the certainty that she knew, without asking them, what they needed. But Robert soon found that ruthless pragmatism got more things done. One early incident is emblematic: deep in the boring sub-clauses of a New York State bill, he buried a radical redefinition of the word “appropriation” – so that the law, once passed, gave the Long Island State Park Commission, which Moses controlled, the power “to write its own laws, hire its own policemen to enforce them and prosecutors to prosecute them”.

At the height of his powers, Moses’s innocuously named Triborough Bridge and Tunnel Authority functioned like a shadow government, with its own flag and police force, a private island headquarters in New York’s East River, staff with access to round-the-clock chauffeurs and, most importantly, its own tax revenue: the tolls that every driver paid to cross the city’s bridges and go through its tunnels. The land Moses controlled in New York State was half the size of New York City. He expanded his influence through a combination of coaxing and cajolery, backroom deal-making, relentlessly long hours, and threats: he retained “bloodhounds” who compiled dossiers on his rivals, documenting any hint of scandal, so a reputation could be swiftly smeared when someone stood in the way of Moses’s plans. (Meanwhile, he brazenly cultivated a public image as a man of total integrity, far above the dirty compromises of politics.) He soon occupied so many crucial government posts simultaneously that he held a trump card: if a mayor tried to restrain him in one area, he simply threatened to resign from all his jobs. He was too popular, and too essential, to be sacrificed; the mayors backed down.

Masterfully, Caro shows how Moses transformed New York in ways both progressive and backward, benign and cruel. Many of the slums he removed were horrendous, and their residents got better homes; he really did break the power of Long Island’s robber-baron estate owners, finally permitting hundreds of thousands of cooped-up middle-income New Yorkers to drive to the beach at weekends. Then again, he so hated the idea of poor people lowering the tone at the seaside that he built bridges over his parkways with insufficient headroom for buses, so only cars could make the trip. Convinced that African Americans had a special dislike of cold water, Caro alleges, Moses kept temperatures in one Harlem pool deliberately low to keep them away. An exceptional chapter, entitled “One Mile”, charts the destruction of a close-knit community by a single, mile-long curve in Moses’s Cross Bronx Expressway – a curve added to the route, Caro strongly suggests, to steer clear of property owned by an influential acquaintance.

When accused of destroying communities, Moses responded, reasonably enough, that a vision like his inevitably meant displacing someone: heed the complaints, and you’d just find yourself facing a different group of naysayers. (“I raise my stein to the builder who can remove ghettoes without moving people,” he remarked, “as I hail the chef who can make omelettes without breaking eggs.”) Far less defensible, along with the racism, was the short-sightedness of his car-centric vision: Moses simply couldn’t comprehend a future in which mass transit, bicycles or walking might play a central role.

Robert Moses with a model of the lower end of Manhattan and the bridge with which it is proposed to connect Battery Park with Brooklyn, March 1939. Photograph: Bettmann/Corbis

His reshaping of New York wasn’t too far advanced before city officials began to notice, with rising alarm, that his new bridges and highways weren’t solving the traffic problem: on the contrary, the creation of more road space seemed only to attract more cars. But Moses stubbornly refused to spend on subways, or to build roads a little wider, so train tracks could run down the middle. If he failed to grasp how hellish New York driving had become, perhaps that was because, for him, it wasn’t: the great evangelist of car culture never learned to drive, and was chauffeured everywhere he went, often on roads cleared in advance by the police.

And yet, despite Caro’s gloomy subtitle, New York didn’t “fall” in any permanent way. When The Power Broker was published, the city was on an accelerating slide toward epidemic levels of street crime, homicide, homelessness and crack addiction; Moses had built gorgeous parks, but you were crazy to visit them after dark. Today, it’s one of the safest big cities in the world, with a burgeoning bicycle culture, an unprecedented expanse of pedestrian zones, and a decent claim to still being the capital of the planet. (On the other hand, the traffic’s still terrible; the motorist-friendly police and prosecutors are still shockingly indulgent toward dangerous drivers.)

The question Caro’s book can’t answer is which of these factors came about despite or because of Robert Moses. Without him, would New York be a faded, economically stagnant ruin, or a big version of Copenhagen? Or perhaps it would be Houston: Moses may have forced through public projects in a high-handed manner, but at least they were public projects; far worse things happen, arguably, when private capital has free rein instead.

In the end, Caro probably overstates Moses’s influence, and understates the victories of his opponents: for instance, a chapter on Jane Jacobs, the urban activist who stopped his plan to drive freeways through Greenwich Village and downtown Manhattan, was removed for reasons of length. But as an account of how power and ambition shape the urban environment, The Power Broker has yet to be beaten. As a resident of New York, I’ve found that reading it changes the way you experience the place: it’s as if you sense the power struggles in every slab of concrete or span of steel. You see how the fixed, physical facts of the city might have been otherwise, had different personalities prevailed. Plus, now, when I’m cursing the lack of decent public transport from the airport, or waiting for a subway train that never comes, at least I have someone to blame.

• To order The Power Broker for £28 go to bookshop.theguardian.com or call 0330 333 6846. Free UK p&p over £10, online orders only. Phone orders min p&p of £1.99.