Anyone who has given birth to a baby, or watched someone else doing it, knows that it's a tight squeeze. The birth canal, framed by the bones of the pelvis, is barely larger than the baby's head. The joint between the two pubic bones at the front of the pelvis softens during pregnancy and can stretch a little during labour. The bones of the baby's skull, separated by fibrous membranes, can overlap a bit as the baby's head makes its way out into the world, which is why so many newborn babies have slightly odd-shaped heads. But even with these concessions, it's a difficult process – much more difficult than in most other animals.

Last December, in the throes of filming a Horizon programme, I flew to New York to meet Holly Dunsworth. I was looking forward to meeting her: she'd authored a paper, published last September, which had caused something of a stir in the anthropological community.

It's always exciting when something like this paper is published. You read it knowing that your worldview will have shifted a little by the time you're finished. I think this is what I love most about science – every now and then something comes along which challenges an existing paradigm, turning ideas on their heads, shaking up your thinking.

The "obstetric dilemma" was an entrenched idea, a neat hypothesis that explained the difficulty of childbirth in humans as well as accounting for another apparently uniquely human phenomenon: our incredibly helpless babies. Whereas chimpanzees are born with brains around 40% of adult size, a human newborn's brain is only 30% of adult size.

The OD hypothesis pitted the demands of walking on two legs against childbirth in an evolutionary tug-of-war over the width of the female pelvis. The idea was that efficient bipedalism – walking and running on two legs, as we humans are wont to do – requires as narrow a pelvis as possible, while the enlargement of the human brain means that we have big-headed babies – which need a wider pelvis to be born. The female pelvis was therefore a design compromise: just wide enough to accommodate a baby's head on its passage to the outside world but nevertheless narrow enough to make childbirth difficult. And babies had to be born "early", while they could still fit through the gap. As for bipedalism, everyone knows men can walk and run faster than women, and they're presumably more energetically efficient at it as well, given their slim-hipped build.

It was an attractive hypothesis but Dunsworth had been suspicious about some of its underlying assumptions. Studying the length of gestation in other primates, she found that human pregnancy didn't seem to be truncated. Then, at a conference, she met up with a couple of other researchers who were looking at the efficiency of bipedalism.

In a lab on the corner of a block in Hunter College, with panoramic views out over New York, Dunsworth introduced me to Herman Pontzer – who co-authored the paper. As Pontzer demonstrated for me with a couple of students walking and running on a treadmill, wearing masks to allow him to measure oxygen consumption, women were just as efficient as men – even with their relatively wider hips.

This finding effectively removed one of the supposed design constraints on the pelvis – it didn't look like there was an advantage in having narrow hips. So why haven't female hips got wider, allowing the baby to stay inside longer?

Before I went to New York I had spent two weeks collecting and freezing small samples of urine after drinking some isotopically labelled water. The samples had been flown to the States, ahead of my trip. Pontzer had analysed my results and now he showed me my data points superimposed on a graph. My metabolic rate was about double what you'd expect – for a non-pregnant woman. But I was five months pregnant: I was providing energy for the developing foetus too. When my metabolic rate rose to 2.1 times the normal rate, I would go into labour – my body would no longer be able to provide the energy being demanded by the foetus.

So it was this "energetic crisis" which seemed to limit the length of pregnancy, not pelvic size. It appears that human babies – like all other placental mammals – are born at a point where the energy demands of the foetus threaten to outstrip the mother's capacity. And at that point, human babies' brains still have a lot of growing and developing to do, simply because our adult brains are so large.

In our ancient ancestors, the pelvis certainly got wider to accommodate bigger-brained babies, but only up to a point – the point at which that energy crunch kicks in. Now mothers the world over may wish that their hipbones were just that little bit wider but it seems that, as far as natural selection is concerned, they're wide enough. And it may be that difficult childbirth is a very recent phenomenon, perhaps due to improved nutrition in affluent societies. In which case, it's a problem which is unlikely to disappear: where women have access to obstetric care, there's no selection pressure against small pelvises. Mothers and babies survive even in cases where a mismatch between the size of the baby's head and the width of the pelvis makes vaginal delivery dangerous or even impossible. The scalpel of the obstetrician is sharper than the scythe of natural selection.