London's Kensington Palace Gardens is the most expensive address in the UK. Its vast mansions are a symbol of the growing gulf between ordinary people and the super-rich. Our reporter tries to meet the locals

It is unexpectedly complicated to write about Britain's most expensive street since its inhabitants, members of a super-rich elite, are not generally willing to give interviews to the Guardian. More problematic still is the hostility which the street's security guards display towards people walking along the road, writing things down in a notepad. Kensington Palace Gardens – where global plutocrats such as Roman Abramovich, Leonard Blavatnik and Lakshmi Mittal own stuccoed mansions, and where one house belonging to a Saudi prince is discreetly on sale (which I think means it isn't on any property websites) for around £100m – is regularly listed as the most expensive place to buy a house in London.

At first glance, there is nothing overtly ostentatious about this quiet road, where the average property was last year valued at around £41m, more than 165 times the value of the average UK home (£248,863). There are no yellow Lamborghinis or Hummers blasting music. The overwhelming impression is one of tasteful reserve, of glistening cream paint and shining green and black railings – until you pause to examine the enormous heft of the houses: vast, detached palaces, with too many windows to count, on a scale dwarfing other private homes in London.

There are armed police officers at both ends of the street, and security huts, where Crown Estate officials control bollards that sink into the ground allowing cars to enter or leave once they have been cleared. It is hard to think of another road in London protected in this way. The need for security is intensified by the presence, just behind the street, of Kensington Palace – home to Prince William, Kate and their baby – and police officers patrol the daffodil-covered lawn in front of the building, followed by sniffer dogs. All of which makes for a rather unfriendly environment; weird and secretive, and slightly reminiscent of somewhere such as Minsk in Belarus, under Alexander Lukashenko's authoritarian rule.

The view from the road on Kensington Palace Gardens. Photograph: Retna/Photoshot

Construction of the street, which runs along Kensington Gardens between Notting Hill and Kensington, began in the 1840s; even then the scale and grandeur of the properties bankrupted one of the first developers, John Blashfield. By 1860 it was already known as "millionaires' row" and was occupied by merchants, landowners and fund managers. These huge houses required large numbers of staff to maintain them; in 1871, the census shows, one house was occupied by a team of 20 servants, tending to their two employers.

Gradually, throughout the 20th century, the houses became unaffordable to run as family homes, even for bankers, and the buildings were sold off as ambassadors' residences. It is only in the last couple of decades, as oligarchs and industrialists have accumulated new extremes of wealth, that the buildings have been transferred gradually back into private hands.

The arrival of this growing roster of private owners, ready to pay unthinkable sums to temporarily own these houses (which have 100-year leases before returning to Crown Estate), has made this street symbolic of the widening gulf that divides London, which itself has become unrecognisable to the rest of the country because of its soaring house prices (up this year by 18%, creating the biggest house price gap between London and the rest of the country since records began).

Does it matter that bits of London feel so removed even from the rest of the capital? Should we just gawp and shrug at excesses like Kensington Palace Gardens? Or should the extremes here trouble us? At the very least, the road is emblematic of how peculiar London has become; a place where some people think nothing of parting with tens of millions to lease a home, while just a couple of miles away others are signing leases to move into garden sheds.

Some anti-poverty campaigners say they don't care too much about the tiny numbers of super-rich, whose lives are so off the scale that they distract from the more pressing problems associated with getting a fairer deal for those on the brink. But it is remarkable how quickly, in the past 20 years, a new top strata of extreme wealth has taken hold again.

If you walk down the street, the houses give little insight into lives of the inhabitants, because their existences are shielded behind net curtains and slatted shutters and privet hedges, which on closer inspection conceal CCTV cameras. More informative are the vans purveying luxury services to the residents.

Eskimo Ice services draws up outside one house – the company delivers ice sculptures for parties, and its website shows glassy ice lions and carved statues of the London skyline. Elsewhere, a van – Anglo-Italian marble installation – is delivering bespoke marble, granite, limestone and porcelain tiles. Gardeners arrive in a van marked Siddeley landscape design (a company that also appears to work on mammoth private estates in China and Russia). British Security Technologies is parked outside another mansion, its van promising in italic lettering: "We'll Keep You Safe 'n' Sound Tonight." A vehicle drives up to provide swimming pool and whirlpool maintenance.

There is also a fire-protection services van, an emergency plumbing car and Rentokil pest control – because, it seems, money offers no real protection against fire, rats and plumbing catastrophes. The plumbing vehicle is outside the Nepalese embassy, which property websites suggest the Nepalese government would like to sell, and which has fallen into a state of disrepair, particularly noticeable next to its expensively maintained neighbours. But someone who lives or works here has put a couple of drooping geraniums on a first-floor windowsill, a touchingly modest, personal attempt at home-making, more human in scale than all the tulips, hyacinths and pansies planted in vast quantities in the gardens along the road, which have been landscaped into luxury-hotel-style anonymity.

While the residents are invisible, their staff can be seen: cleaning golf clubs with a hose in front of one house; wearing a black waistcoat and ironed white shirt to polish a bronze door handle; walking tiny dogs.

Some of the houses verge on hideous, iced with yellow-cream stucco, shiny, bright and reminiscent of custard or, at the Kensington end of the road, like dreary, red-brick school buildings. There are menacing stone eagles on gateposts and disconcertingly huge flower pots, large enough to conceal a crouching human. Some of the houses look uninhabited.

But parts of the road are lovely: the Narnia-style lampposts, and the avenue of plane trees; the lack of pollution allows the scent of frangipani from a garden to linger in the air. There is so little noise that birdsong is audible in a way that is quite rare in London. There is no graffiti and no rubbish, because a street sweeper is using a machine to blow leaves into piles.

The signs on the doors are excessively polite, and use outmoded words such as "kindly" and residing". "Kindly do not deliver items for Mr and Mrs [...] to this address as they are no longer residing here." But it is the doorbell etiquette that is most enraging, and instructions that "for all collections and deliveries please press the housekeeper's button only" incite a sudden surge of anarchic rage and a desire to ring all the other bells simultaneously – summoning the chef/kitchen, the residence and the caretaker.

Later I wonder if the lampposts have been wired up to CCTV because my slow progress down the road, making notes of doorbell instructions, has not gone unnoticed. Outside the Israeli embassy, which is surrounded by protective green bollards, an armed police officer stops me. "Madam, what are you doing?" he asks, his gun slung across his chest. I explain that I am writing about the street.

Kensington Palace Gardens … weird and secretive. Photograph: Oli Scarff/Getty Images

"Do they know that you are doing this?" he asks.

He tells me the road is owned by the Crown Estate and it would be better if I went to one of the security booths to get permission to walk down the street with a notepad. I explain that I don't feel inclined to do that.

"Did you tell them what you are doing? It would be courteous to let them know. This is a private road."

I ask if he is telling me that I have to go and register with the security officials, and after some thought he decides he won't make me turn back. "I'm not telling you that you have to. I'm just saying that it would be the courteous thing to do."

I wonder a bit about a Metropolitan police officer giving lessons about courtesy, and move on.

About 45 minutes later, when I am back outside the Nepalese embassy at the other end of the road, I hear a voice buzzing out of the guard's handset, describing a "lady with brown-coloured hair, shoulder length". I'm asked for the second time what I'm doing, and informed that the Diplomatic Police Corps has asked me to leave the road. "You shouldn't be walking down the road. The Crown Estate gives permission for people to walk but we can stop anyone from coming down here," a guard explains, so I leave. Diplomats and oligarchs must appreciate the vigour with which security guards approach their role.

Mostly, residents here decline emailed requests to talk about what it is like to live here. It is uniquely difficult to report on the lives of the very rich because they are, as a rule, not anxious to be forthcoming. Someone in Lakshmi Mittal's vast entourage emails that he does not want to talk about moving into the street. An assistant to Jon Hunt – who sold Foxton's estate agents for £390m in 2007 and who owns a vast, gloomy, grey mansion here – replies to say that he rarely grants interviews and adds: "This is not something he would participate in." (His house, the subject a dispute over the merits of his plan to build an underground sports centre and museum for his vintage car collection, has been left unoccupied since he bought it from the Russian embassy in 2005, but a tour of the inside is available online, as the backdrop to a peculiar motorbike stunt video.)

Tamara Ecclestone's agent rejects a request to talk about cultural attitudes towards wealth in this country, explaining: "Sadly I will have to decline this request, whilst Tamara is appreciative of her blessed life she doesn't feel talking about money is appropriate. Best wishes …" Which is a little odd, because Hello! readers have been given detailed insights into the redevelopment work on her £45m home here, which she (apparently) updated with an underground bowling alley, dog spa and £1m bath carved from Amazonian crystal. ("But I spend a lot of time in the bath so it's worth it.'')

Tim Wright, from estate agents Knight Frank, who handles sales over £10m, has sold three houses on the street over the past decade, but will not give details of who to or how much for. "Like a lot of big houses the type of ownership has changed. There was a period in the middle of the 20th century when people were not able to maintain these big houses, but with the explosion of wealth generation globally in the last 20 years, the use has changed back to family houses."

He says his clients are not concerned by spending huge amounts of money for leaseholds properties, commenting simply: "Mine is not to reason why." But he thinks the prices (of around £6,000 per sq ft) are justified because Kensington Palace Gardens "is regarded as one of the most exclusive addresses in the world. That's not just hyperbole."

Kensington Palace Gardens … the guarded entrance. Photograph: Alamy

Nicholas Foulkes, partner at Colwyn Foulkes architecture, has submitted plans for a new building on the site of No 3, for the owner of a global construction company. The new house – the first to be built in a century – will cost around £18m to construct and may be worth up to £40m once completed (since it is at the more modest end of the street where houses are a little smaller).

None of which sheds much light on what it is like to live on London's most expensive street, until help comes from the Finnish ambassador, Pekka Huhtaniemi, the next-door neighbour of Leonard Blavatnik, the Ukrainian-born American businessman. His perspective, given over a herring breakfast in the ambassadorial residency's cavernous reception room, is interesting because it is coloured with the bemusement that someone steeped in Finland's more egalitarian principles feels about Britain's extremes of wealth.

Huhtaniemi moved here from his relatively modest 150sq m apartment on the outskirts of Helsinki in 2010. He likes to keep the vast building full of visiting jazz musicians and students and artists, and feels a responsibility to ensure that it is used to the maximum as a working space. He and his wife have a flat at the top of the house (which seems enormous but which he describes as the smallest on the street). Despite appreciating this is a remarkable building, he lists a number of disadvantages of living on London's most expensive street.

Aside from the fact that the buildings are extraordinarily draughty, badly designed for insulation (by Finland's high standards) and wasteful of energy, the street is peculiar because of the marked absence of any sense of community. "You don't know your neighbours anyway these days, but no I wouldn't say that this is a community. The diplomats, we meet each other, we go to each other's receptions, we are sometimes invited by colleagues to attend a dinner or a working luncheon. But the wealthy businessmen, the millionaires and billionaires, they keep a very low profile," he says.

Then there's the sense of isolation which comes with not knowing if the buildings around you are actually lived in, or are simply investment vehicles.

"The palaces are often dark at night. You don't know if people keep lights on as a precaution, in order to perhaps to keep the burglars away," he says. "Now of course you have to be a very brave burglar to come here, through all the checkpoints …"

The constant noise from the extravagant, wholesale refurbishment underway in at least five of the street's houses is also a significant annoyance. Like residents of all of London's most expensive areas, he has been particularly disturbed by the noise and dust involved in comprehensively gutting and remodelling houses, and digging out multilayered basements.

"Mr Blavatnik has been refurbishing his house, which is next to us, for three years. I hope that will be finished next year. In the early days when they started, they did a lot of drilling. Then it was very noisy, let's say between 7am and 5pm. They have not worked around the clock, not in the evenings. They are working inside the house now, for the last six months it has been pretty quiet …"

The only time Huhtaniemi met Blavatnik and Mittal was when Kensington Palace organised a garden party for the neighbours to apologise for the noise and disruption caused by their own building works in 2011. "They wanted to make a friendly gesture, so that was in 2011. That's the only occasion when it has been possible to meet the neighbours in one go."

Finnish visitors are always impressed at the house, and never question their government's decision to keep it (the lease ends in 2040); visitors mainly want to know whether he has seen Kate pushing her baby in a pram along the pavement. "I haven't."

The extreme wealth on display frequently makes him thoughtful about the meaning of money, and he is clear that this is not a place he would choose to live in the longer term.

"You come to the basic philosophical question: does money make you happy? I personally don't think that money alone makes you happy. If you have a lot of money, you also have a lot of worries. You have to worry about your security much more, you have to worry about your investments. You can wake up in the morning and when you look at the share prices, you suddenly see that you have lost £50m overnight. I'm not envious of very rich people. I think that they have their own share of worries."

But he understands the street's attraction for the super-rich.

"Rich people want to stick together; to be close to each other. This is why they probably buy property in the same Alpine resorts – Courcheval in France, St Moritz in Switzerland. They feel comfortable when they are with their own social equals," he says. "There is a saying in Finnish: money is a little bit like snow. It tends to gather in heaps."

• This article was amended on 8 April 2014. It originally said the Finnish ambassador had lived in a 150sq ft apartment, rather than a 150sq m one.