Choosing not to have children is selfish, or so says the pope, who is surely something of an expert on this one, being childless himself. Reminiscing this week about his own upbringing as one of five, he told an audience in St Peter’s Square of his fears for a “greedy generation that doesn’t want to surround itself with children, that considers them above all worrisome, a weight, a risk”.

Children, he argues, bring hope to society, rejuvenating and enriching all our lives. Perhaps they do. It’s just a shame the same cannot always be said of their parents.

Jonathan Hall, a police community support officer from Cambridgeshire, shot briefly to fame this week after describing school-run mothers as “wailing and hissing” feral cats with hate-filled eyes and a complete disregard for double yellows, hell-bent on blocking the roads for everyone else. Roughly around the time the pope was speaking, I was listening on the radio to a discussion about youth rugby coaches forcing parents to watch matches in silence because too many were barracking other people’s offspring from the touchline or swearing at the ref. Perhaps there is more than one way of contributing to the impoverishment of society.

This isn’t just about Catholicism, and its vested (some might even say selfish) interest in church growth by encouraging the faithful to go forth and multiply. We live in an age where one in five women will not have had children by their mid-40s. (And no, there’s no equivalent data for men; giving birth to a child being a more unmistakably public event than fathering one). For some that will have been a positive choice, for others painfully not so; and for a third group it’s perhaps somewhere in the middle – a difficult consequence of choices made but not regretted, a fork in the road not taken. But although it’s something millions experience, not having children still carries a wholly unfair stigma for both sexes, an unspoken sense that you owe the world an explanation.

The greatest taboo of all remains confessing to be child-free by choice

Any woman in the public eye who doesn’t have children can expect endless nosy questions about why it never happened; whether she had to choose between baby and career, or whether perhaps there’s an intimate gynaecological problem that she fancies sharing with the world.

But increasingly, childless men are viewed with suspicion. Remember that faintly desperate Gordon Brown photo opportunity, arranged in the days before he had children of his own, involving him at the birthday party of a small child of a staff member? Borrowing someone else’s kid was clearly preferable to the risk of being portrayed as a workaholic loner, incapable of sharing his life.

And the greatest taboo of all remains confessing to being child-free not by accident but by choice. Stories of infertility, miscarriage or just never having found the right person do at least evoke sympathy. The bile heaped on the former Australian prime minister Julia Gillard for being what her critics called “deliberately barren” is a reminder of how often the mature, considered choice not to have children is portrayed as cold and heartless.

It’s true that parents sacrifice sleep, sanity and the distant memory of having once had plans of their own in return for stinking nappies, a houseful of plastic tat and the chance to provide a free taxi service for grunting ingrates. In a strict economic sense, our children are a public good; falling fertility rates in Italy, as in other countries, have shrunk the tax base alarmingly at a time when workers must support historically unprecedented numbers of older people through their retirement. (Although evidence from Scandinavia, where fertility rates are markedly higher than in the traditionally Catholic south of Europe, suggests that providing good cheap childcare and getting men to share the parenting does more to encourage large families than a scolding from the church.)

Having a baby doesn’t have to mean retreating into your own tiny, self-absorbed world, either. Many do find parenthood a time of powerfully awakened civic feelings and sudden urges to sign petitions or wave placards – even if they are often for causes in our children’s interests – and worry about global warming.

But anyone claiming to have given birth for the benefit of a grateful world is either a liar or an ocean-going narcissist. Having children is without doubt the single most selfish thing I’ve ever done: I did it not for the greater good of a planet that is already teeming with more humans than it can sustainably support but frankly because I wanted to, and because it makes me happy.

There’s precious little evidence that having children makes you a nicer person – arguably in the sleep-deprived years, it just makes you a grumpier one – and certainly no evidence that bludgeoning people into having children they don’t really want helps anyone concerned.

In generations gone by, too many men and women grimly knuckled down to producing the family that was seemingly expected of them, closing the door on other choices that might have made them happier. Some of them made a reasonable job of it, but others merely raised their children to feel obscurely burdensome, unwanted, guilty for reasons that they never really understood. For anyone who knows themselves well enough to know they wouldn’t suit parenthood, avoiding it isn’t selfish but wise and arguably compassionate.

The big puzzle is why Pope Francis – a man who turned his back on family life voluntarily to serve what for him is a higher cause – seemingly can’t imagine others feeling the same; can’t see that there are other callings. If it’s right for priests and nuns to give themselves wholly to God, then what is wrong with devoting your life to brain surgery, or teaching, or founding a charity or developing a business? Or, indeed, to none of the above, but to being a good citizen and friend; to leading a happy, meaningful, productive life?

Childlessness should worry us where it’s involuntary, or where it serves – as in the upper echelons of the City, or at the height of recessions – as a canary down the mine: a telltale sign of working environments so relentless or social conditions so tough that having children feels impossible.

Everywhere else, frankly, it’s no one else’s business.