Immigration: it should be so hard to talk about, and yet it is so very easy. Raise the subject, as the government does ham-fistedly almost every week, and the debate that follows is always the same. First the left refers to “dog-whistle” politics. Then the right insists that these conversations, far from being racist, are “necessary”. Finally, our newspaper columnists, liberal and ­otherwise, make futile attempts to advance the argument with full reference to (delete as applicable) their Syrian dry-cleaner/their friend who teaches at an inner-city state school/their friend who has a child at an inner-city school/their former career as a teacher at an inner-city state school/terrorism/the white working class.

But perhaps this is unfair. For what, in the end, does any of us know about all this? The world of the immigrant lies beyond our range of vision, invisible and unreported. We live here, and he lives . . . there. Although we flatter ourselves that we sometimes visit his realm – didn’t we take an Uber the other day, and wasn’t the driver chatty? – the truth is that we do not see it at all, not close up. Almost every idea we have about it is preconceived, the result of second- or third-hand information. This is a choice: we turn away from what troubles us. But its more desolate expanses would resist our attention even if we were to venture in. We might as well be the police or the immigration service, for all the welcome we’d get.

It is this land, secretive and lonely, that Ben Judah seeks to reveal in his frantic, teeming new narrative, This Is London. Judah had long sensed that there was a parallel universe out there; like anyone who has lived in the capital for more than a decade, he had seen with his own eyes the shift in its population (over a third of Londoners were born abroad, half of them having arrived since 2000). But uncovering it was never going to be simple. His book does not involve idle conversations with some Aussie barista down at Starbucks, and it relies not at all on cosy meetings fixed up by charities and community associations. His quest is deeper than this. He wants to get into all the ugly, unremitting stuff: the details of what it might be like to live eight to a room, or in a freezing-cold underpass that stinks of piss and sweat; to be trapped, in effect, in a city whose doors were only reluctantly opened to you in the first place (or weren’t opened to you at all, should you be one of the 600,000 people living in London illegally). How to do this? When he can, Judah plays the journalist: he wins people’s trust and listens to what they tell him. When he can’t, when those skills prove to be less than useless, he plays the immigrant, squeezing in for a night in the subway below Hyde Park Corner beside a beggar called the Fiddler, sharing a room in Barking with a man called Baby, who rises at 5am each morning to drill solar panels illegally into a concrete base somewhere over the M25 – a cash-in-hand job that’s a two-hour commute by bus.

“I know what you want,” a Romanian, the Interpreter, says to him at one point. “You want the same fucking Mo Farah wonder story . . . Well, let me tell you what . . . you’re ain’t gonna find it, bro.” In fact, this man is wrong. Judah is not interested in dreams come true. He’s chasing disillusion.

Like the city in which its subjects live, this is a crowded book, and a noisy one. Its voices compete for our attention. Builders, cleaners, butchers, drug dealers, gambling addicts, runaways, prostitutes: they can all be found inside it. So, too, can a copper who came to London from Nigeria in 1989 and whose job it is now to police Peckham; a Polish registrar who sometimes has to send couples she believes to be attempting to marry illegally smartly on their way (she knows she’s right if, as they leave, the man lets the door swing in the girl’s face); and a Nigerian teacher who works at a school in Plaistow, in the East End, where 80 per cent of the students speak English as a second language. Their stories unspool slowly at first but have a fiercely unstoppable quality once they get going. This isn’t a read for the timid or the politically correct. The interviewees’ casual racism – right at the bottom of the list of those many of them despise are the “ugly, lazy”, white British working classes – is just as shocking as anything you’ll have heard on Celebrity Big Brother.

Thanks in part to London’s immigrants, the city’s topography has changed, too, and when he is not busy talking, or listening, it is to this that Judah turns his eye. Once-proud suburbs – Pooterish, but nicely kept – such as Neasden and Harlesden are shabby now, their pebble-dashed semis and red-brick villas divided, and then divided again, rooms within rooms. Off Seven Sisters Road in north London, an old priory has become a mosque, decked in electric green fairy lights. On the Old Kent Road – he walks it 18 times, country-coding its shopfronts – only one old pub remains (there used to be 12). Meanwhile, places out east such as Beckton, a Brookside-ish fantasy redeveloped in the 1980s for a middle class that preferred to go elsewhere, sink into deprivation, their various communities carefully segregated from each other. “I don’t live in Britain,” a woman tells him. “I live in Lithuania. I watch Lithuanian TV . . . I use Lithuanian internet . . . My friends are all Lithuanian . . . I only meet Lithuanians. The only thing I do in Britain is pay taxes to the British.”

As Judah notes, nowadays the pretend place that is Walford, supposedly west of this point, looks like nothing so much as a bit of creepy nostalgia.

How to feel about this odd, bristling book? I think it is deeply flawed. Sitting somewhere between the embedded, granular reporting of Barbara Ehrenreich’s exposé of low-wage America, Nickel and Dimed, and the unmediated voices of Craig Taylor’s Londoners, the narrative would have made for a less confusing prospect if Judah had edged slightly in the direction of the former by being clearer about his process. He flits around, and though this reflects the transience inherent in the immigrant experience, it grows wearying for the reader. Reporting and listening, though closely related, are not the same thing. Judah takes a great deal on trust. The book is also overwritten in places. When he described a branch of Waterstones as “crusty” and used the term “Bo-Peep” as an adjective to describe a terraced house, I wasn’t entirely sure what he meant. Some of his characters – an Egyptian heiress with a bad case of ennui, the wife of a not-quite-an-oligarch Russian – seem to belong in another book altogether.

Yet somehow, This Is London rises, and then soars, above such (petty?) criticisms. Work of this sort really is necessary; this is the stuff we must think about if we are ever to get to grips (assuming it’s not too late ­already) with what lies ahead for our cities. Every MP should be given a copy immediately. On every page lies an uncomfortable truth, in every paragraph sheer horror. It is a book that demonstrably improves the eyesight. Read it, and the streets will look different: I guarantee it. Above all, more than I can possibly say, I admired its author’s pluck, determination, compassion and refusal to judge – and I’d like him to know that some of the stories he told will haunt me for a long time to come.

This is London: Life and Death in the World City by Ben Judah is published by Picador (352pp, £18.99)