The average British male is 5ft 10in and will tell you that he's about 5ft 11in, while the average woman is 5ft 4in and will probably apologise for it. That last thought's guesswork, but the rest is fact, or at least a well-sourced Wikipedia entry. How tall we are, according to new research by the clinical psychologist Prof Daniel Freeman, correlates closely with how confident we feel. Freeman is interested in the relationship between paranoia and a sense of inferiority. In his experiment his subjects were fooled by virtual reality into thinking for part of the time they were surrounded by people taller than themselves. It showed there was a strong connection between being small, or at least smaller than the people around you, and feeling small, between the perceived physical reality and the internal emotional reaction.

There are obvious evolutionary reasons for feeling safer when you're bigger than the likely opposition. But it's aeons since the brain triumphed over brawn when it came to getting hold of power and keeping it. Yet we are still apparently conditioned to regard taller people with the kind of respect that comes from an atavistic fear that they might thump us, and it's so persistent that tall people, it is claimed, get better jobs, earn more and generally have a better time of it than their shorter brothers and sisters.

Until a generation or so ago, there was definitely a class thing going on. In the mid-18th century, the difference between officer recruits (always upper class since joining the army took more ready cash than most people saw in their lifetimes) and the ordinary soldier was an astonishing 7.5in. Even in 1950 there was still a three-inch difference between manual and non-manual workers. But that doesn't quite explain Freeman's results, which were about confidence, and anyway there are plenty of people around who aren't tall doing very well thank you, including those whose appearance is at least part of their work, such as the actor Daniel Radcliffe.

Film is particularly adept at blurring the difference. Size matters only where it is relative. It seems obvious: you can't feel small when you are taller than everyone in the room. Wrong. Speaking as a taller than average 5ft 8.5in, you can. Or at least I could as a tall child in a single-sex school, perpetually stooped in an attempt to make eye contact with my friends, always taller than the rest of my class. Except then feeling small was actually feeling tall, being different – wearing glasses and having a mortifying tendency to knock things over – and not in a good way. What I most wanted in life was to be able to do a backflip like my lithe best friend. Playing Romeo because I was tallest was not really compensation.

Yet that discrimination may be at the heart of it. It's not that tall people feel superior, or more effective, or demand to be talked to with particular deference. It's just how shorter people treat them. For a start, you literally have to look up to the taller person. And the taller person has to look down. Tall people get more space because they take more space. Maybe that reads across into their attitude to life. But maybe the tall person isn't taking an advantage. They're being given it. Just like the people in the experiment.

This is the emotion Gloria Steinem identified on the streets of Tokyo when, analysing an unexpected sense of ease, she realised that – for the first time in her life – most men were shorter than her. As she says, we aren't in the habit of recognising how our sense of our selves in physical space shapes the way we think and forms part of our mental wellbeing. It is not just our mind speaking to our body: it's our body speaking to our mind.

No one can make you feel inferior without your consent, Eleanor Roosevelt once said. Somehow, it seems significant that Mrs Roosevelt was 5ft 11in.