If scientists from Mars were to study the human male's reproductive system they would probably conclude that he is destined for rapid extinction. Compared to other mammals, humans produce relatively low numbers of viable sperm – sperm capable of making that long competitive swim to penetrate an unfertilised egg.

As many as one in five healthy young men between the ages of 18 and 25 produce abnormal sperm counts. Even the sperm they do produce is often of poor quality. In fact only between 5 and 15 per cent of their sperm is, on average, good enough to be classed as "normal" under strict World Health Organisation rules – and these are young, healthy men. By contrast, more than 90 per cent of the sperm of a domestic bull or ram, or even laboratory rat, are normal.

Human males also suffer a disproportionately high incidence of reproductive problems, from congenital defects and undescended testes to cancer and impotency. As these also affect fertility, it's a minor miracle men are able to sire any children at all. In fact, an increasing number of men are finding themselves childless. Among the one in seven couples now classed as infertile, the "male factor" has been found to be the most commonly identified cause.

Next year marks the 20th anniversary of the WHO conference where a Danish scientist first alerted the world to the fact that Western men are suffering an infertility crisis. Professor Niels Skakkebaek of the University of Copenhagen presented data indicating sperm counts had fallen by about a half over the past 50 years. Sperm counts in the 1940s were typically well above 100m sperm cells per millilitre, but Professor Skakkebaek found they have dropped to an average of about 60m per ml. Other studies found that between 15 and 20 per cent of young men now find themselves with sperm counts of less than 20m per ml, which is technically defined as abnormal. In contrast, a dairy bull has a viable sperm count in the billions.

Experts in human reproductive biology were astonished by the Danish study. The declining trend seemed to indicate that men were on a path to becoming completely infertile within a few generations (although recent studies suggest the fall in sperm counts may have bottomed out). Professor Skakkebaek could offer no explanation for the trend other than to suggest that the fall may have something to do with the equally alarming rise in other reproductive disorders, such as cancer of the testes and cryptorchidism, the incomplete descent of the testes into the scrotum.

Experts began to talk of a new phenomenon affecting the human male, a collection of disorders known as testicular dysgenesis syndrome. They wanted to know what was causing it, because the changes were occurring too quickly to be a result of genetics. It must have something to with changing lifestyles or the environment of men, and almost everything was suggested, from exposure to chemical pollutants to the modern fashion for tight underpants. There is now an emerging consensus among some experts that whatever it is that is exacerbating the problems of male infertility, it probably starts in the womb. It is not the lifestyle of men that is problem, but that of their mothers.

The process of sperm production, called spermatogenesis, starts in adolescence, but the groundwork is laid down in the few months before and immediately after birth. An increasing number of studies point to a crucial "window" of testicular development that begins in the growing foetus and ends in the first six months of life. Interfere with this critical developmental period, and a baby boy will suffer the lifetime consequences of being a suboptimally fertile man.

So are we anywhere nearer to finding an explanation for why are so many more men today are suffering from reproductive problems?

"It's most likely a reflection of the fact that many environmental and lifestyle changes over the past 50 years are inherently detrimental to sperm production," says Professor Richard Sharpe, fertility research expert at the Medical Research Council. "It may be that different factors come together to have a combined effect." A number of studies point to a connection between early development in the womb and male reproductive problems in later life, especially low sperm counts. For example, men whose pregnant mothers were exposed to high levels of toxic dioxins as a result of the 1976 industrial accident in Seveso, Italy have been found to have lower-than-average sperm counts. But men exposed to dioxins in adulthood showed no such effect. Another study found women who ate large amounts of beef during pregnancy, a diet rich in potentially damaging chemicals called polycyclic aromatic hydrocarbons (PAHs), had sons with relatively low sperm counts. But eating beef as an adult man shows no similar impact.

Meanwhile, studies of migrants between Sweden and Finland, showed that a man's lifetime risk of testicular cancer tends to follow the country he was born in rather than the country where he was brought up. It was his mother's environment when she was pregnant with him, rather than his own as a boy or as an adolescent, that seems to have largely determined a man's risk of testicular cancer.

One of the strongest pieces of evidence in support of this idea comes from studies of people who smoke. A man who smokes typically reduces his sperm count by a modest 15 per cent or so, which is probably reversible if he quits. However, a man whose mother smoked during pregnancy has a fairly dramatic decrease in sperm counts of up to 40 per cent – which also tends to be irreversible.

Professor Sharpe said such findings can be explained by understanding how the first cells of the testes form. Sertoli cells, which in the adult act as guardians for the development of sperm cells, are the very first cells to form from a "genital ridge" of the human male foetus. The number of sperm that can be produced in an adult man is critically dependent on the number of Sertoli cells that develop in his foetus, so anything that interferes with the formation of Sertoli cells in a mother's womb will affect sperm production many years later. "Maternal-lifestyle factors in pregnancy can have quite substantial effects on sperm counts in sons in adulthood, and the most logical mechanism by which this could occur is via reducing the number of Sertoli cells," Professor Sharpe says.

But the key question now is to identify the relevant lifestyle and environmental factors.

This is proving tricky. Obesity, for instance, is a growing problem and it has been linked with reproductive problems in both men and women. One study has also indicated that overweight pregnant women tend to produce sons with poor semen quality. But is it being fat that is the cause, or the environmental chemicals stored in fat?

There has been a lot of interest in chemicals in the environment, especially those that can either mimic female sex hormones – oestrogenic chemicals – or block male sex hormones, specifically testosterone which plays a critical role in stimulating the development of Sertoli cells in the womb. So far, the Seveso study provides the clearest link between human foetal development, low sperm counts and prenatal exposure to an environmental chemical. But the dioxin concentrations from this industrial accident were exceptionally high.

It is more difficult trying to establish a similar, significant link between male reproductive problems and exposure to low concentrations of the many other environmental chemicals that may have weak oestrogenic or androgen-blocking properties, including substances as wide-ranging as pesticides, traffic fumes, plastics and even soya beans. Professor Sharpe says that much of the evidence to date is weak or non-existent.

"Public concern about the adverse effects of environmental chemicals on spermatogenesis in adult men are, in general, not supported by the available data for humans. Where adverse effects of environmental chemicals have been shown, they are usually in an occupational setting rather than applying to the general population," he says.