Zachary Scott for TIME

There are a lot of downsides to being male. We age faster and die younger. But give us this: we’re lifetime baby-making machines. Women’s reproductive abilities start to wane when they’re as young as 35. Men? We’re good to go pretty much till we’re dead.

The reason, of course, is sperm: Unlike ova, they’re hardy and decidedly plentiful. Every 16 days or so the male body raises a whole new army of them hundreds of millions strong. Want to use a few of those reproductive foot soldiers to keep conceiving children far into your fifties, sixties and even seventies? Have at it, and I should know: I didn’t have my children until I was in my mid- and late-40s (for more, read my story in the new issue of TIME, available to subscribers here).

But not so fast. Older fathers, it turns out, can present as many medical problems as older mothers—more in fact. For all the concerns about Down syndrome and other genetic disorders that become more common in babies of older mothers, the list of conditions older fathers bring to the table is turning out to be far longer. Just last year, a study in Nature found that rates of autism and schizophrenia rise sharply in the babies of older dads, with the risk doubling for every 16.5 years of paternal age. Another study, also in Nature, found something similar for autism, beginning when a man is just 35—the same ostensible trouble-age as for moms. Yet another paper in the American Journal of Men’s Health linked paternal age to preterm birth and low birth weight, and others have found connections to cleft lip and certain cancers.

The problem arises from the same 16-day turnover rate that make sperm such an infinitely renewable resource. Every batch of sperm represents an opportunity for genetic typos—called de novo mutations—to be passed on. A 20-year-old man and woman will each pass on about 20 de novo mutations to a baby they conceive. By the time the couple is 40, a woman’s total has remained at 20, while a man’s has jumped to 65—and it keeps climbing from there.

(MORE: The Secret to Super Sperm? Less TV)

Then too there are the social issues associated with paleo-fatherhood. There aren’t many gray heads among the fathers in my daughters’ play groups but mine. On at least one occasion when I picked the girls up at school, a child asked me, “You’re the dad?” which I found mildly funny, but the girls didn’t. And I’m actually at the very young end of the old dad cohort. Paul McCartney had a baby when he was 61; Rod Stewart was 66; Rupert Murdoch was a stunning 72. Not only does that mean they’ll have less stamina than the average dad, that means they’ll, well, check out a lot sooner too.

“Even if you’re Paul McCartney’s child, you get ripped off if your father dies when you’re in your early 20s,” says Julianne Zweifel, a clinical professor of obstetrics and gynecology at the University of Wisconsin.

Older fatherhood isn’t all bad: testosterone rates drop about 1% per year as men age, making them less reactive and more patient, and a professionally established middle-aged man is likely to have more time and money to devote to his kids than a twenty-something who’s just getting started. What’s more, it’s good for moms too—in a satisfying way. After all the generations, even centuries, women have spent under the medical microscope as they go about the simple business of trying to make a baby, it’s nice to have dad—too often a free-rider in the procreation game—take a little of the heat too.

MORE: Older Fathers Linked to Kids’ Autism and Schizophrenia Risk