Prof Colin Ellard was walking past the rows of new-build towers that dominate the west of central Toronto when he had a sudden realisation. “I was struck by how dark, sombre and sad these new urban canyons made me feel,” he says.

Ellard, a cognitive neuroscientist at the University of Waterloo in Canada who studies the impact of places on the brain and body, wanted to know why he felt like that – and if others felt the same.

His curiosity ultimately led him to conduct a series of virtual reality experiments in which he asked people to wear specialised headsets and stroll through a variety of urban environments created to test their responses. The findings, he says, proved he was not alone. Being surrounded by tall buildings produces a “substantial” negative impact on mood.

Facebook Twitter Pinterest One of Tom Ryaboi’s photographs of the Toronto streets from the top of a skyscraper. Photograph: Tom Ryaboi/Barcroft Media

If proven, Ellard’s theory adds weight to existing studies finding a negative effect of high-rises on the mental health of city residents. With both government policy and the potential for greater profits driving high-density construction in cities around the world, this raises an important question for the development industry.

Urban environments are associated with higher levels of psychiatric disorders (pdf) – city dwellers have a 40% increased risk of depression and double the rate of schizophrenia, according to the Centre for Urban Design and Mental Health. Ellard’s idea is that the moment to moment bad feelings he observed in the virtual reality environment can affect everyday interactions in the real world and people’s experience of living in cities.

“When people are in these very dense environments that produce oppressiveness and increase negative emotion, it seems logical that those things will spin off into the ways we understand other people and the way we treat them,” he says. “Those are the variables that are most likely to show relationships with [increased incidence of] psychiatric illness.”

Can prefab homes solve UK's housing crisis? Read more

This may seem a big leap but Ellard is the latest in a long line of researchers to see a link between high-rises and poor mental health. Nicholas Boys Smith, founder of built environment social enterprise Create Streets, analysed academic studies on high-rise living for his 2016 report on the design of cities. According to Boys Smith, the balance of evidence shows residents of high-rise blocks tend to suffer from more stress, mental health difficulties and neurosis, with child development particularly affected. “High-rise can work, but it’s much harder,” he says.

This all appears to cut against the urban planning orthodoxy that a certain level of density – around 30-50 homes per hectare – is necessary to make lively communities that are able to support shops, businesses and public transport. This idea is the reason the government endorsed higher density development close to transport links in February’s housing white paper.

These concerns also come up against the commercial reality in high value areas such as London, where there are more than 400 towers in the development pipeline (pdf). David Birkbeck, chief executive of housing consultant Design for Homes, says inflated land prices mean developers have little option but to go upwards to make their investment back. “Once they’ve outbid everyone for a site, height is their only way to recover the price paid,” he says.

The question is how to build densely without these negative repercussions. “The villain isn’t density itself, it’s insensitive design,” says Layla McCay, director of the Centre for Urban Design and Mental Health. “It’s about how you design in things that are protective to people’s mental health – green spaces and opportunities for social interaction.”

Some have concluded there is a density “sweet spot” (pdf) that gives the benefits of sustainable city living without the mental health costs. Proponents of mid-rise development such as that found in European cities like Vienna and Barcelona, for example, argue for buildings constructed to heights of up to eight storeys within mixed use neighbourhoods where residential buildings sit alongside shops, offices and other work spaces.

“Living at one storey is probably the most healthy thing for the human animal. But it is so much worse for the environment as a whole. That’s why we talk about the sweet spot,” says architect Jason McLennan, founder of the US-based Living Future Institute.

London's changing skyline: planned tall buildings 'almost double in two years' Read more

At the same time Boys Smith says there are signs the commercial impetus that has encouraged tower building may be altering, particularly given the recent reduction in demand for high-end housing in London. While towers are more profitable, the greater up front cost of construction makes them riskier at times when demand isn’t as strong.

Boys Smith says he is working with one large UK developer who is considering reworking unbuilt tower schemes in its pipeline using a “mid-rise” approach, and he and others promoting a mid-rise approach sense an opportunity, particularly where developers haven’t overpaid for land. Hackney council has also been promoting a similar idea for the controversial Bishopsgate Goodsyard site on the edge of the City of London, where developers Hammerson and the Ballymore group have gone back to the drawing board on earlier proposals to build seven towers of between 17 and 46 storeys.

The incoming president of the Royal Institute of British Architects, housing architect Ben Derbyshire, says many developers are already “reacting quite well” to the idea. “It’s perfectly possible at mid-rise density to design places with plenty of biodiversity, with a mix of uses and well-designed streets,” he says.

Boys Smith admits that having a home at all is the most important thing. Likewise a recent report by homelessness charity St Mungo’s shows the number of homeless people in the capital with an identified mental health need has tripled in five years. But that doesn’t make concerns about densification invalid. “It’s about finding a more humane way to house people when we do it,” says Boys Smith.

Sign up to be a Guardian Sustainable Business member and get more stories like this direct to your inbox every week. You can also follow us on Twitter.