"We thought he was mad, but he had a lot of money and wanted us to find out what was going on," Fitzpatrick recalls.

Over the next months, Fitzpatrick carried out an exhaustive study of soya and its effects. "We discovered quite quickly," he recalls, "that soya contains toxins and plant oestrogens powerful enough to disrupt women's menstrual cycles in experiments. It also appeared damaging to the thyroid." James's lobbying eventually forced governments to investigate. In 2002, the British government's expert committee on the toxicity of food (CoT) published the results of its inquiry into the safety of plant oestrogens, mainly from soya proteins, in modern food. It concluded that in general the health benefits claimed for soya were not supported by clear evidence and judged that there could be risks from high levels of consumption for certain age groups. Yet little has happened to curb soya's growth since.

More than 60% of all processed food in Britain today contains soya in some form, according to food industry estimates. It is in breakfast cereals, cereal bars and biscuits, cheeses, cakes, dairy desserts, gravies, noodles, pastries, soups, sausage casings, sauces and sandwich spreads. Soya, crushed, separated and refined into its different parts, can appear on food labels as soya flour, hydrolysed vegetable protein, soy protein isolate, protein concentrate, textured vegetable protein, vegetable oil (simple, fully, or partially hydrogenated), plant sterols, or the emulsifier lecithin. Its many guises hint at its value to manufacturers.

Soya increases the protein content of processed meat products. It replaces them altogether in vegetarian foods. It stops industrial breads shrinking. It makes cakes hold on to their water. It helps manufacturers mix water into oil. Hydrogenated, its oil is used to deep-fry fast food.

Soya is also in cat food and dog food. But above all it is used in agricultural feeds for intensive chicken, beef, dairy, pig and fish farming. Soya protein - which accounts for 35% of the raw bean - is what has made the global factory farming of livestock for cheap meat a possibility. Soya oil - high in omega 6 fatty acids and 18% of the whole bean - has meanwhile driven the postwar explosion in snack foods around the world. Crisps, confectionery, deep-fried take-aways, ready meals, ice-creams, mayonnaise and margarines all make liberal use of it. Its widespread presence is one of the reasons our balance of omega 3 to omega 6 essential fatty acids is so out of kilter.

You may think that when you order a skinny soya latte, you are choosing a commodity blessed with an unadulterated aura of health. But soya today is in fact associated with patterns of food consumption that have been linked to diet-related diseases. And 50 years ago it was not eaten in the west in any quantity.

In 1965, the earliest year for which the Chicago Board of Trade keeps figures, global soya bean production was just 30m tonnes. By 2005, the world was consuming nine times that a year, at 270m tonnes. World soya oil production, meanwhile, has increased sevenfold over the same period, from 5m tonnes to 34m tonnes a year.

To feed demand, new agricultural frontiers are being opened up in Brazil, where large areas of virgin rainforest have been illegally felled to make room for the crop. US-based transnationals are now exporting soya back to China, the country from which it originated, as newly urbanised Chinese switch to industrialised western diets. Thanks to US agribusiness, we have developed an apparently insatiable global appetite for the bean produced by farmers in the Americas.

James and Fitzpatrick became convinced early on that this entirely new dependence on soya was, in fact, a dangerous experiment. The dead parrots were no joke - they were the canaries in the coalmine.

For James and his wife Valerie, breeding the exotic birds down under was a retirement dream. They wanted to feed their young birds the best, so they began giving the chicks a soya feed. Parrots do not eat soya beans in the wild but the high-protein animal feed had been marketed in the US as a new miracle food.

The result was a catastrophic breeding year. Some of the birds were infertile; many died. Other young male birds aged prematurely or reached puberty years early. "We realised there was some sort of hormonal disruption going on but we'd eliminated other possible hormone disrupting chemicals such as pesticides from the inquiry," Fitzpatrick says.

So the toxicologist began a systematic review of the scientific literature on soya. After finding out about the plant oestrogens in soya, Fitzpatrick says, "My next thought was: what about children who are fed soya milk?" He calculated that babies fed exclusively on soya formula could receive the oestrogenic equivalent, based on body weight, of five birth control pills a day.

In fact, it had been known since the early 1980s that plant oestrogens, or phyto-oestrogens, could produce biological effects in humans. The most common of these were a group of compounds in soya protein called isoflavones. Food manufacturers had variously marketed soya foods as an antidote to menopausal hot flushes and osteoporosis, and as a protective ingredient against cardiovascular disease and hormone-related cancers. Large quantities of mainly industry-sponsored scientific research have been produced to back up these claims. The American soya industry spends about $80m every year, raised from a mandatory levy on producers, to research and promote the consumption of soya around the world. The rash of new soya foods can be seen as the latest in a line of innovative ways devised to use soya.

The hypothesis behind the health claims is that rates of heart disease and certain cancers such as breast and prostate cancer are lower in east Asian populations with soya-rich diets than in western countries, and that the oestrogens in soya might therefore have a protective effect.

Fitzpatrick, however, looked into historic soya consumption in Japan and China and concluded that Asians did not actually eat that much. What they did eat tended to have been fermented for months. "If you look at people who are into health fads here, they are eating soya steaks and veggie burgers or veggie sausages and drinking soya milk - they are getting over 100g a day. They are eating tonnes of the raw stuff."

Mass exposure to isoflavones in the west has only occurred in the past 30 years due to the widespread incorporation of soya protein into processed foods, a fact noted by the Royal Society in its expert report on Endocrine Disrupting Chemicals in 2000. When the independent experts on the scientific committee on toxicity trawled through all the scientific data, they concluded that soya milk should not be recommended for infants even when they had cow's milk allergies, except on medical advice, because of the high levels of oestrogenic isoflavones it contains.

On breast cancer, they decided that "despite the suggested benefits of phyto-oestrogens in lowering risk of developing breast cancer, there is also evidence that they may stimulate the progression of the disease". The lower risk of certain cancers among Asian populations might be due to other factors - their high consumption of fish, for example. They advised caution. On the effects on menopause symptoms, the evidence was inconclusive, the experts ruled. On bone density, the committee thought there might be some protective effects, but the data was unclear. The evidence on prostate cancer was mixed. Since isoflavones cross the placenta, the implications of pregnant women eating large quantities of soya were unclear. There was some evidence that soya-based products had a beneficial effect on the good HDL cholesterol but they were not sure that was down to the isoflavones. On the other hand - reassuringly - they judged that a study linking soya consumption to decline in cognitive function was not convincing.

What the committee also pointed out was that the way soya was processed affected the levels of phyto-oestrogens. Traditional fermentation reduces the levels of isoflavones two- to threefold. Modern factory processes do not. Moreover, modern American strains of soya have significantly higher levels of isoflavones than Japanese or Chinese ones because they have been bred to be more resistant to pests. (One way to tackle pests is to stop them breeding by making them infertile. It turns out that unfermented soya did play one role in traditional Asian diets - it was eaten by monks to dampen down their libido.)

Sue Dibb, now food policy expert at the National Consumer Council, was a member of the CoT working group that compiled the final report. She questions whether infant soya milk should still be on public sale and is troubled by the latest marketing of soya. "We looked in detail at the claimed health benefits for adults for soya consumption and concluded there was not sufficient evidence to support many of them. There may be benefits but there are also risks. The groups of adults of particular concern are those with a thyroid problem and women with oestrogen-dependent breast cancer. It worries me that soya is being pushed as a health food by a big soya and supplements industry. We ought to be taking a more cautious approach."

The Food Standards Agency advice is that soya's potential to have an adverse effect on babies' hormonal development is still controversial, but that soya formula should only be given to infants under 12 months old in exceptional circumstances.

Professor Richard Sharpe, head of the Medical Research Council's human reproductive sciences unit at Edinburgh University, was also a member of the committee's working group on phyto-oestrogens in food. He has been studying the decline in male fertility in the past half-century. He recently completed studies on the effects of soya milk on young male monkeys which showed that it interferes with testosterone levels. "In the first three months after birth, baby boys have a neonatal testosterone rise. The testes are very, very active in hormone production at this point and there is a lot of cell activity going on that will determine sperm count in adults and will affect the developing prostate. If you introduce a phyto-oestrogen, which can, in large amounts, alter these changes, you may predispose children to later disease. Soya formula milk is a [recent] western invention. There is not the historical evidence to show it is safe."

Manufacturers, however, argue that soya infant formula has been widely used without problems. "The industry has said that if the CoT comes up with clear science, we will take note, but the case is not proven," says Roger Clarke, director general of the industry's Infant Dietetics Food Association. "A lot of the work it looked at was based on experimental work with animals. There does not seem to be clear evidence of adverse effects, and there is demand for it. There are some markets, such as vegan usage, where soya is the only alternative."

While 30-40% of all infants in the US are raised on soya formula - not least because it is given away in welfare programmes - soya milk for babies has always been confined to a small minority in the UK. So does Sharpe think exposure to soya from other sources - vegetarian soya proteins, the soya flour in factory bread, the hydrolysed proteins added as flavourings, for example - has a cumulative effect that might be worrying to other age groups? He says he is not concerned about people who eat soya foods in moderation or in the way they are traditionally used in oriental diets, but when it comes to modern processed foods, which use soya proteins in different ways, he prefers to turn the question round. "If someone said they were adding a hormone to your foods, would you be happy with that? There may be lots of effects, some of them may be beneficial, but would you be happy with that? I am not a fan of processed foods, full stop. And these quick fixes for protecting against ill-health - you know they can't be true," he adds.

A steaming hiss fills the kitchen of the top London restaurant Nobu, even after the lunchtime rush. Japanese chefs are filleting the evening's fish while stock bubbles and concentrates in its stainless steel vat behind. Executive chef Mark Edwards hands me a teaspoon of one of his soy sauces. Cool from the fridge, it is thick, rich, dark and sweet, yet remarkably clear from its long fermentation. The miso that he uses to marinade his famous black cod for three days is dense and strong from its lengthy brew too. Muslin cloths envelop delicate curds of tofu, made fresh each day and added in small cubes to miso soup.

Soya is used in traditional oriental diets in these forms, after cultures, moulds or precipitants have achieved a biochemical transformation, because in its raw form the mature bean is known not only for its oestrogenic qualities but for also its antinutrients, according to the clinical nutritionist Kaayla Daniel, author of The Whole Soy Story. Soya was originally grown in China as a green manure, for its ability to fix nitrogen in the soil, rather than as a food crop, until the Chinese discovered ways of fermenting it, she says.

The young green beans, now sold as a fashionable snack, edamame, are lower in oestrogens and antinutrients, though not free of them. But raw mature soya beans contain phytates that prevent mineral absorption and enzyme inhibitors that block the key enzymes we need to digest protein. They are also famous for inducing flatulence.

Christopher Dawson, who owns the Clearspring brand of organic soy sauces, agrees. He lived in Japan for 18 years and his Japanese wife, Setsuko, is a cookery teacher. "I never saw soy beans on the table in Japan - they're indigestible."

Dawson describes the traditional craft method of transforming the soya bean through fermentation, so that its valuable amino acids become available but its antinutrients are tamed. The process involves cooking whole soya beans, complete with their oil, for several hours, then adding the spores of a mould to the mix, and leaving it to ferment for three days to begin the long process of breaking down the proteins and starches. This initial brew is then mixed with salt water and left to ferment for a further 18 months, during which time the temperature will vary with the seasons. The end result is an intensely flavoured condiment in which the soya's chemical composition has been radically altered. Traditional miso is similarly made with natural whole ingredients, slowly aged.

Most soya sauces (and misos) are not made this way any more, however. Instead of using the whole bean, manufacturers short-cut the fermentation by starting with defatted soy protein meal. Soya veggie burgers and sausages generally use the same chemically extracted fraction of the bean.

This meal is the product of the industrial crushing process the vast majority of the world's soya beans go through. The raw beans are broken down to thin flakes, which are then percolated with a petroleum-based hexane solvent to extract the soya oil. The remains of the flakes are toasted and ground to a protein meal, most of which goes into animal feed. Soya flour is made in a similar way.

The oil then goes through a process of cleaning, bleaching, degumming and deodorising to remove the solvent and the oil's characteristic "off" smells and flavours. The lecithin that forms a heavy sludge in the oil during storage used to be regarded as a waste product, but now it has been turned into a valuable market in its own right as an emulsifier.

In so-called "naturally brewed" soya sauces the processed soy protein meal is mixed with the mould spores and given accelerated ageing at high temperatures for three to six months. Non-brewed soya sauce, the cheapest grade, is made in just two days. Defatted soya flour is mixed with hydrochloric acid at high temperatures and under pressure to create hydrolysed vegetable protein. Salt, caramel and chemical preservatives and flavourings are then added to provide colour and taste. This rapid hydrolysis method uses the enzyme glutamase as a reactor and creates large amounts of the unnatural form of glutamate that is found in MSG.

Most commercial soya milk today is made from soya isolates, although some of the pioneers of soya foods as health products in Europe avoid the chemical extraction process and use whole beans to make their milk. The key selling points for both types of soya milk are that they contain complete proteins and oestrogenic isoflavones.

Bernard Deryckere, president of the European Natural Soyfood Manufacturers Association, says that his members' products, made using natural processes, are a healthy alternative to diary products. "A lot of people in Europe are lactose-intolerant. Soya milk was invented in China 4,000 years ago and today it's consumed by all types of people as a cholesterol-free source of quality protein."

Daniel's detailed examination of the history of soya milk, however, suggests that soya milk was made not to drink, except in times of famine, but as the first step in the process of making tofu. After the long, slow boiling of soya beans in water to eliminate toxins, a curdling agent was added to the liquid to separate it. The curds would then be pressed to make tofu and the whey, in which the antinutrients were concentrated, would be thrown away.

Dibb points out that if you are drinking non-dairy milk because you want calcium without cow's milk, there are plenty of other sources such as green leafy vegetables and nuts. And only those eating extremely limited diets are likely to be short of protein as adults.

Dawson, a lifelong vegetarian, does not drink soya milk and only eats tofu in moderation. "I will only use a product for my family if there is 200 years of tradition behind it. You are asking for trouble if you take an isolate from soya - yet so much effort seems to go into taking industry's waste and turning it into new food."

The effort that has gone into creating the global soya market has indeed been enormous. Today it is dominated by a handful of American trading companies. Three of them - Bunge, ADM and Cargill - control 80% of the European soya bean crushing industry. These three, together with allied companies, are also estimated to control up to 80% of European animal feed manufacturing. They dominate the US soya market, and also account for 60% of Brazil's soya exports.

Before the first world war, only a very few soya beans were crushed. The Americans had begun experimenting with using the protein meal as animal feed, but farmers were reluctant to take it up because it was indigestible to chicken and pigs. The oil produced was considered "a bit of an embarrassment", according to Kurt Burger, a fats and oils technical expert at the Society of Chemical Industry, whose experience in the food industry goes back to 1944. It was mainly used in soaps because it was considered unpalatable. (Henry Ford later funded research projects to turn soya into plastic for car parts.)

Cottonseed oil, a byproduct of the cotton industry, was the main edible oil used in the US. But then the combination of disease in monocropped cotton and demand from European allies in the first world war for oil both to eat and to make the glycerine needed for nitroglycerine in explosives, stimulated American soy oil production.

It was not until the 1940s that industry worked out how to deactivate the enzyme inhibitor in the protein meal sufficiently for animals to tolerate it, and it was only technology taken from the Nazis at the end of the second world war that solved the problem of the oil's horrible smell and flavour. That left the way for the US to promote the soya that suited its agricultural conditions as part of the reconstruction of Europe through the 1950s. Soya oil exports to Europe tripled under the Marshall Plan, and heavily subsidised exports of surplus US soya ensured the commodity's dominance in animal feed. The subsidies continue. Between 1998 and 2004, US Department of Agriculture figures show that its soya farming received $13bn in subsidies from the American taxpayer.

Until 2003, the US was the largest exporter of soya. But through the 1990s, multinationals promoted the expansion of the crop in Latin America, helping finance farmers and building the infrastructure for soya exports. The attraction of Latin America is that land is cheap and labour costs are minimal too. Three years ago, the combined exports from Brazil and Argentina surpassed US exports for the first time. The cost is now being counted there in environmental damage and social upheaval. The cost to western consumers may yet be counted in health.