After the weather, there are few things Londoners like to complain about more than the high cost of either buying or renting a place to live. To say London is an expensive place to live is to state the obvious. However, one or two examples will serve to show just how bad the situation has become.

According to the BBC housing calculator the average price of a house now stands at a staggering £402,000 in Greater London. This is around 2.5 times more than the average for Scotland and well over £100,000 more than the next closest region (unsurprisingly the South East). These prices make buying a home in London extremely difficult – if not impossible – for families on modest or even not-so-modest incomes.

Examining the situation from a global perspective doesn't make the situation look any better. According to Global Property Guide, London is second only to Monaco in terms of the cost of buying or renting a place to live. While other studies don't place London quite so high on the list, no one claims that London is cheap.

A recent 0.2% drop in housing prices has benefited those who can afford to buy a place of your own, but rents continue to increase. Over the last 12 months London rents have risen by 4.5% (2% above the current inflation rate) to an average of £1,032 per month.

Shelter claims that a family needs a gross income of around £52,000 a year to affordably rent a 2 bedroom flat at the median rental price. Moreover, high rents have led to all kinds of horror stories of tenants living in sheds and sub-par conditions.

Of course, London is a huge and diverse city with rents and property prices varying widely by area. To see how much flats cost to rent in your postcode, have a look at the London rents map.

The Usual Suspects

Given the situation most Londoners find themselves in, who are we to blame? All sorts of explanations have been given as to why these high prices exist and persist. Two common villains in this saga are greedy landlords charging as much as possible for tiny shoebox-sized flats (or worse) and greedy property developers who only build flats and houses that cater to the well-to-do.

Other common explanations are the number of foreign buyers who operate in the London market, lack of social housing, and planning restrictions.

While each of these elements plays a large role in distorting the market, there is one factor that is often overlooked – but is the one I think goes further than any other in explaining the reason for London's high prices. However, you'll have to keep reading to find out what it is. Before I get to it, I want to briefly examine some of the other commonly cited causes for London's insane rental and housing situation.

Ultimately, the price of most things comes down to supply and demand. Buying a house or renting a flat is no different, except that there are so many restrictions around both that the market is severely distorted. Some of these distortions, such as social housing, do provide huge benefits to those that receive them, but also come at a cost which is often not addressed.

However, before we get in to the supply side of things, let's first look at demand. Overall, it is pretty clear that demand is outstripping supply. Demand is strong for two primary reasons: one, London's economy is performing better than the rest of the UK's. Two, there is a steady stream of foreign buyers. It is the latter that I want to look at as foreign buyers are often blamed for the high prices seen in the London market.

I should point out that when talking about foreign buyers, I am referring specifically to those who come from overseas to buy houses and flats more as investments than as places to live. Immigrants to London should not be included in this because they both live and work here. A third of Londoners were born outside the UK, and as a result, comprise roughly a third of the number of people looking for housing.

Obviously, the fact that there are so many immigrants like me means that there is increased demand for housing. Yet, London could not function without its immigrant labour force.

The problem with foreigners who buy property – but don't live here – is that they take that space away from people who could both live and work here. Thus, the housing stock diminishes from what it could be. It also means that London not only has to cater to the wealthy of the UK, but to the wealthy of the whole world.

Realistically, trying to close off London's property market from foreign buyers would probably not work . I'm sure it would have many unforeseen consequences and goes against a city that has depended and thrived on trade throughout its existence. But, with a report from Savills saying that over half of all prime London property is being purchased by foreign buyers, you'd be naïve to think this segment of the market doesn't have a huge impact on the rest.

In fact, I think it's a major reason why you see private rental costs increasing the way they have recently. The seemingly never-ending financial crisis has meant that foreign buyers are seeking out places to buy as a safe haven. Over the last few years, the pound has declined sharply against other currencies, which makes London property seem relatively cheap if your wealth is tied up in euros.

Couple this with general instability in many markets and you can see why some buyers choose London. If these economies improve or the pound rises sharply, you can expect foreign buyers to be a drain on property prices as they sell out. This means the level of foreign buying may end up being a temporary blip rather than a long-term trend. In many – if not most cases – it's not greed that's driving these buyers, but fear about even worse economic conditions back home.

Bankers, Supply and Council Flats

Another aspect of the financial crisis that has impacted the London property market is that banks are no longer as willing to lend people money. Banks should be more stringent than they were during the former boom times, but this means people must stay in the rental market awhile longer since they can't afford to buy. Still, when house and rental prices increase, potential first time buyers get squeezed at both ends and saving for a deposit becomes next to impossible.

However, we must keep in mind that demand is still only half the equation. The other half is, of course, the supply of housing stock – both to rent and buy. Interestingly, nearly 1/4 of all London households rent a property from one of the 33 councils. People living in these houses and flats are charged below-market rents. As anyone who has taken a basic economics course will tell you, the result is that there are far more people looking to get into social housing than there is space available.

Unfortunately, this means that a huge number of people are left outside the council housing system. This creates a system of winners and losers among people who may be equally deserving. It also encourages people who've made it into the system trying to cheat it.

Now, I may seem negative towards the council housing system, but I don't entirely mean to be. I think it is a very interesting idea and one that clearly benefits millions of Londoners. The problem is that I don't think it can solve the supply issue.

The Real Cause - Not Enough Housing Density

Housing density is the reason for London's high rental and property prices. Specifically, London lacks a sufficiently high concentration of houses and flats in areas most people want to live in. Far more people want to live centrally than can afford to do so. Even though there are now almost no free plots of land anywhere in zones 1 or 2, people still want to move there and so you get overcrowding along with insane property valuations.

As an example, many viewers reacted negatively to the portrayal of landlord Andrew Panayi in episode 3 of The Secret History of Our Streets: Caledonian Road. For those who haven't seen the episode, Panayi owns a large section of Caledonian Road and is portrayed as trying to cram as many people as possible into tiny flats. While many found his methods very suspect, the wider problem is that he is meeting market demand in the only way possible, given current planning restrictions.

London is a very low-rise city, which makes it very liveable and walkable, two things I love about it. Yet, this makes it very unique among cities its size. In the developed world, most cities the size of London build upwards to meet demand. London hasn't done this and even recent census figures show that London is far less dense than other comparable cities.

So let's first look at London; Which borough has the highest population density? According the 2011 census, Islington is now the densest local authority in the UK, overtaking Kensington and Chelsea. 206,100 people live in 5.74 square miles. This gives a population density figure of around 36,000 people per square mile. It may seem like a lot of people in a small area, but when compared to other cities internationally it's actually quite low.

London Compared To Tokyo And New York

First let's look at Tokyo – the city that I suspect for most, defines density. Nakano Ward is the densest of the 23 wards of Tokyo, with around 52,000 people per square mile. Moreover, 14 of Tokyo's wards have higher population density figures than Islington. However, for true density we can look just the other side of the Atlantic. Manhattan has nearly 1.6 million people living in around 23 square miles of space. This gives it a population density of 71,000 people per square mile. This is even more amazing when you consider that a large portion of the global financial industry is located on the island, along with a huge amount of other offices and commercial buildings.

If all of London was built to the same density of Manhattan, it would be by far the largest city on earth, housing over 43 million people. Interestingly, this is roughly the number of people in England living outside London today.

Of course, London is neither New York nor Tokyo. Manhattan was forced to build up because it's an island. Moreover, the fact that a large portion of the island is granite means that it's relatively easy to build tall buildings. However, Tokyo's managed to achieve a higher level of population density than London even though it's located in the most earthquake-prone part of the world. The fact that London can build buildings like the Shard shows that there is no strong technical reason why we can't build up.

High-Rise Living Problem?

Unfortunately for London renters and many buyers, London's first experiment with high-rise living largely turned out to be a failure. In my opinion, this has more to do with the experiment itself rather than the concept behind it. Many people won't of course see it that way. The 1960s and 1970s saw the rise of vast concrete council estates filled with (relatively) high-rise buildings. More often than not, they were built according to planner fiat rather than market demand.

Added to this is the problem that most were built at a time when architectural style reached its lowest point; it turns out that most people don't want to live in Brutalist concrete blocks. Moreover, most of these eyesores were cut-off from the surrounding community by design. The idea was to create a closed community. The inevitable result was no-go areas which attracted crime and poverty; ironically the type of thing they were trying to replace.

However, there are some notable exceptions from this time period. Concrete blocks located in desirable areas have seen property values increase. Two examples in London include the Barbican Estate and the Trellick Tower. Properties in the Barbican can now go for over £1.5 million. However, this has more to do with location than architectural merit.

Most Londoners today are at best wary and at worst downright hostile towards high-rise buildings. London currently ranks 37th in terms of the number of buildings over 100 metres tall (just behind Caracas, Venezuela). Despite this, London is the 18th largest metropolitan region in the world.

Solutions?

London has some amazing skyscrapers that are recognised around the world, like the Shard, the Gherkin, and the Strata building. The thing is we need more of them and we need people to live in them if we want to meet the demands of residents today and tomorrow.

Many people no longer want to live out in the suburbs or in new towns. Instead, they want to live in walkable communities in urban environments that cater to their wants and needs. The only way this demand can be met is by building up in the areas in which people most want to live.

That said, I don't think we should return to the planning and architectural disasters of the 60s and 70s. High-rise buildings must be fully integrated into the streets they surround and not be cut-off in dead-end, no-go estates. They should be mixed in among existing low-rise buildings to make sure that areas don't go too far in any one direction. Politicians often say we need to build more houses and flats, but what they don't normally tell us is that it will have to be a certain type of housing, if it is to meet current demands.

There is of course the option of continuing London the way it is: a low-rise city with the central areas increasingly only affordable to those on City incomes or to wealthy investors from abroad. However, if this is the choice it should be made with the full realization that the average family will be unable to afford to buy or rent a home in central London, unless they are "lucky" enough to be allocated a council flat or house.

The only other alternative is the Panayi approach; keep subdividing flats into smaller and smaller units, pushing them where they were not designed to go. If people are willing to pay £100 a week for a room in an unventilated basement or in an unheated shed, it shows that there is a demand that is not being met by current housing options.

The Economist's, solution is to build in the Green Belt. "Taking a mile off the Green Belt all around London would release around 25,000 hectares, the equivalent of a sixth of London's area." I disagree on their proposed solution, but I have to give them a hand for advocating a bold and likely to be extremely unpopular idea. At least they are trying to come up with something.

However, once you set the precedence of building in the Green Belt, it becomes difficult to argue against further development. Moreover, the price of housing seems to indicate people want to live in more central locations, not further out.

My current borough of Kensington and Chelsea is an interesting case in point. It is the only borough in London to have a population decline between 2001 and 2011. In 2001, it was the densest local authority in the UK, and remains second in 2011. During that period house prices more than doubled. With no new houses to meet ever rising demand, prices increased far faster than in other parts of the city.

Supply and demand are the true causes of London's high property and rental prices, not greed. There is too much demand in areas where there is not enough supply. Demand is high for a variety of reasons, but the real constraint at this point is supply. Both planners and the public as a whole seem to hate the idea of building up, due in no small part to the failures of earlier attempts at high-rise living.

This is entirely understandable, but unfortunate as it means the symptoms rather than the root cause are blamed. "Greedy" landlords are just responding to a demand that cannot be met at present. It is for that reason that councils and private developers should look to high-rise, high-density living as a serious option for housing across all income levels. Not all of London should be high-rise, but more of it needs to be if we want to meet the housing demand over the next 50 or 100 years.