MEDIEVAL noblewomen swallowed arsenic and dabbed on bats' blood to improve their complexions; 18th-century Americans prized the warm urine of young boys to erase their freckles; Victorian ladies removed their ribs to give themselves a wasp waist. The desire to be beautiful is as old as civilisation, as is the pain that it can cause. In his autobiography, Charles Darwin noted a “universal passion for adornment”, often involving “wonderfully great” suffering.

The pain has not stopped the passion from creating a $160 billion-a-year global industry, encompassing make-up, skin and hair care, fragrances, cosmetic surgery, health clubs and diet pills. Americans spend more each year on beauty than they do on education. Such spending is not mere vanity. Being pretty—or just not ugly—confers enormous genetic and social advantages. Attractive people (both men and women) are judged to be more intelligent and better in bed; they earn more, and they are more likely to marry.

Beauty matters most, though, for reproductive success. A study by David Buss, an American scientist, logged the mating preferences of more than 10,000 people across 37 cultures. It found that a woman's physical attractiveness came top or near top of every man's list. Nancy Etcoff, a psychologist and author of “Survival of the Prettiest”, argues that “good looks are a woman's most fungible asset, exchangeable for social position, money, even love. But, dependent on a body that ages, it is an asset that a woman uses or loses.”

Beauty is something that we recognise instinctively. A baby of three months will smile longer at a face judged by adults to be “attractive”. Such beauty signals health and fertility. Long lustrous hair has always been a sign of good health; mascara makes eyes look bigger and younger; blusher and red lipstick mimic signs of sexual arousal. Whatever the culture, relatively light and flawless skin is seen as a testament to both youth and health: partly because skin permanently darkens after pregnancy; partly because light skin makes it harder to hide illness. This has spawned a huge range of creams to treat skin in various ways.

Then again, a curvy body, with big breasts and a waist-to-hip ratio of less than 0.8—Barbie's is 0.54—shows an ideal stage of readiness for conception. Plastic surgery to pad breasts or lift buttocks serves to make a woman look as though she was in her late teens or early 20s: the perfect mate. “Mimicry is the goal of the beauty industry,” says Ms Etcoff.

Basic instinct keeps the beauty industry powerful. In medieval times, recipes for homemade cosmetics were kept in the kitchen right beside those used to feed the family. But it was not until the start of the 20th century, when mass production coincided with mass exposure to an idealised standard of beauty (through photography, magazines and movies) that the industry first took off.

From small roots to big business

In 1909, Eugène Schueller founded the French Harmless Hair Colouring Co, which later became L'Oréal—today's industry leader. Two years later, Paul Beiersdorf, a Hamburg pharmacist, developed the first cream to bind oil and water. Today, it sells in 150 countries as Nivea, the biggest personal-care brand in the world. Around the same time, in Tokyo's upmarket Ginza, Arinobu Fukuhara hit on eudermine lotion—the first Japanese cosmetic based on a scientific formula, and the first product for the Shiseido company.

But it was the great rivalry between two women in America that made the industry what it is today. Elizabeth Arden opened the first modern beauty salon in 1910, followed a few years later by Helena Rubinstein, a Polish immigrant. The two took cosmetics out of household pots and pans and into the modern era. Both thought beauty and health were interlinked. They combined facials with diets and exercise classes in a holistic approach that the industry is now returning to.

Rubinstein considered facelifts (via leather straps and electricity) to be as acceptable as lipstick, while Arden pioneered beauty branding, with her iconic gold and pink packaging. The two women, together with Max Factor (which originally produced make-up for actresses), built the foundations of modern marketing, bewitching consumers with aggressive tactics such as celebrity endorsements and magazine advertorials. In the 1930s they were joined by Revlon, and after the second world war by Estée Lauder. All these companies are still around.

The emerging beauty industry played on the fear of looking ugly as much as on the pleasure of looking beautiful, drawing on the new science of psychology to convince women that an inferiority complex could be cured by a dab of lipstick. Even then, ruthlessness and outright quackery lurked behind the façade. On launching her famous eight-hour cream, developed for her horses, Arden quipped: “I judge a woman and a horse by the same criteria: legs, head and rear end.”

Anything but skin-deep

Analysts at Goldman Sachs estimate that the global beauty industry—consisting of skin care worth $24 billion; make-up, $18 billion; $38 billion of hair-care products; and $15 billion of perfumes—is growing at up to 7% a year, more than twice the rate of the developed world's GDP. The sector's market leader, L'Oréal, has had compound annual profits growth of 14% for 13 years. Sales of Beiersdorf's Nivea have grown at 14% a year over the same period.

This growth is being driven by richer, ageing baby-boomers and increased discretionary income in the West, and by the growing middle classes in developing countries. China, Russia and South Korea are turning into huge markets. In India, sales of anti-ageing creams are growing by 40% a year, while Brazil has more “Avon Ladies” (900,000) than it has men and women in its army and navy. Although the industry's customers are predominantly women, it is increasingly marketing itself to men too.

The juicy returns are attracting new entrants. The household-goods giants Unilever and Procter & Gamble (P&G), facing maturity in many of their traditional businesses, are devoting more resources to their beauty divisions—as evidenced by P&G's current $6.5 billion offer for Germany's Wella, a hair-care company, to bolster its earlier purchase of Clairol, a hair-dye business. Simon Clift, marketing head of Unilever's personal-care businesses (including its big Dove and Sunsilk brands), says: “We are a Cinderella, seen as a soap-powder company, but we have changed our belief about what we do well.”

And so to bed

Most luxury-goods groups now have perfume brands, and many (like Dior, Chanel and Yves St Laurent) are selling make-up and creams too. LVMH, the biggest luxury-goods group of all, has moved into retailing with its Bliss spas and Sephora shops (which sell make-up).

At the same time the industry is consolidating. Many innovative younger brands have been swallowed up by the giants. Japan's Kao bought John Frieda to tap into the hair-dye business, one of the fastest-growing segments of the market. And in the past five years, LVMH has bought Hard Candy and Urban Decay, two funky young make-up brands; while Estée Lauder has acquired Stila, MAC and Bobbi Brown, another collection of up-and-coming names in make-up.

Since this burst of transactions, six multinationals account for 80% of American make-up sales, while eight brands control 70% of the skin-care market. With its Nivea brand, Beiersdorf is one of the few large independents left, desired by everyone from P&G and Unilever to L'Oréal.

Science fictions

Unable to outspend their big new rivals, the traditional beauty companies are trying to out-innovate them, by making even more of their scientific credentials. The industry is marketing a new category of products that blurs the line between cosmetics and non-prescription drugs—so called “cosmaceuticals”. L'Oréal's advertisements now stress how many product patents it has filed.

The focus on science has led to some genuinely new ideas, such as face cloths impregnated with cleansers that combine surfactant and paper technology. As yet, though, most of it is pseudo-science. Shiseido's recent international launch of its new Body Creator skin gel claims that its fat-burning pepper and grapefruit oil can melt 1.1kg of body fat in a month without any need to diet or exercise. At last year's launch in Japan, customers bought a bottle every 3.75 seconds.

Avon's chief executive, Andrea Jung, expects Cellu-Sculpt, a new cream that claims to take an inch off your thighs in four weeks, to sell three times as much as an ordinary body cream in its first six weeks. P&G is also playing the game. It is busy plugging the science behind its newest cream, Olay Regenerist, and it has built Pantene into the world's biggest hair-care brand on the basis of its “pro-vitamin B” ingredient. But a report by Britain's Which? magazine recently pointed out that vitamins need to be ingested to work, and that a Pantene shampoo it tested was no better than a supermarket's own brand.

That is hardly surprising: Jacques-Franck Dossin, an analyst at Goldman Sachs, says that beauty firms spend just 2-3% of their sales on research & development—compared with 15% by the pharmaceuticals industry. On the other hand, they spend a whopping 20-25% on advertising and promotion. Some of that money has been well spent. L'Oréal's “Because I'm worth it” tag-line has long been a huge success. And new companies like Pout are attracting attention with lipsticks labelled “Lick my lolly” and “Bite my cherry”.

Nevertheless, marketing is becoming hugely expensive, putting pressure on margins. Goldman's Mr Dossin worries that, in response to intensifying competition and consolidation, L'Oréal has recently cranked up its spending on marketing, forcing an advertising war with rivals that will end up undermining everyone's profitability. Scott Beattie, Elizabeth Arden's boss, says that its marketing budget, which grew by 25% in 2002, will rise by another 40% this year. Avon plans to hike its advertising budget by 50%.

This competition has left some exhausted. Revlon, once one of the biggest make-up brands, has been tottering on the edge of bankruptcy. Its current boss, Jack Stahl, a former president of Coca-Cola, is fighting to get the business back on track. Unilever, which is facing slowing growth overall, is stranded in no-man's-land. It sold the great Elizabeth Arden brand and missed out on the boom in hair colour. Critics say it should now sell its remaining beauty business to a more focused group, such as Elizabeth Arden. Meanwhile P&G, seen as the first serious threat to L'Oréal for decades, is struggling with its $5 billion acquisition of Clairol. It has been losing market share at an alarming rate to both the French and to Kao's John Frieda.

Changes in distribution are also helping to separate the winners from the losers. The only real growth is coming through huge grocery chains such as Wal-Mart that want to deal with just a handful of big suppliers. That is good news for P&G and L'Oréal (which already gains two-thirds of its revenues from mass retailers). But Estée Lauder and Revlon are more dependent on unfashionable department stores where sales are declining and selling costs are high. Estée Lauder's chief executive, Fred Langhammer, bravely preaches the virtues of department stores: “People will always need advice about what skin cream to use,” he says. But he is wisely hedging his bets by buying fun specialist retailers, such as MAC.

Body rebuilding and refitting

Two potentially lucrative markets are being all but ignored by the traditional beauty companies. The first is cosmetic surgery, already a $20 billion business, which has been growing and innovating by leaps and bounds. The number of cosmetic procedures have increased in America by over 220% since 1997. Old favourites, such as liposuction, breast implants and nose jobs, are being overtaken by botox injections to freeze the facial muscles that cause wrinkles. With the number of these up by more than 2,400% since 1997, botox injections have become the most common procedure of all.

The newest lines are bottom implants, fat inserts to plump up ageing hands, and fillers like Restylane and Perlane for facial wrinkles. Cosmetic dentistry is also a booming business. Jeff Golub, Manhattan dentist to stars like Kim Catrall of “Sex and the City”, dubs himself a “smile designer”. “We are able to create all sorts of illusions,” he says. “The smile has become a fashion statement.” Tooth whitening is the botox of the cosmetic dentistry business.

What used to be the preserve of actresses and celebrities has become safer and more affordable. Alan Matarasso, one of America's leading plastic surgeons, says: “Ten years ago you could reconstruct a woman's breasts for $12,000—now it can be done for $600.” Drooping prices have helped cosmetic surgery to move into the mainstream. More than 70% of those who come under the knife now earn less than $50,000 a year.

The second big new market is in “well-being”—whole treatment systems that cover beauty, exercise and diet, including visits to spas, salons and clubs, and hark back to the early days of Mesdames Arden and Rubinstein. People are increasingly seeking natural cures rather than turning to chemicals, and an emphasis on being fit—not just thin—is growing in popularity. The trend is being led by a list of celebrities. Avon's boss, Andrea Jung, says modern beauty has been “redefined as health, self esteem and empowerment.”

Beauty firms are falling over themselves to sell products with new-age promise—Arden has a range of products called “Happy”, Avon sells diet bars, and L'Oréal owns a few spas. However, it has been left largely to entrepreneurs like American cosmetic surgeon Stephen Greenberg to offer real innovations. His “extreme make-overs” combine cosmetic surgery, a personal trainer for your body, and an image consultant for your face and hair. The traditional beauty companies have yet to grasp the opportunities in these rapidly growing and fragmented markets.

A slap in the face?

At the same time, the beauty business needs to guard against a growing consumer backlash. Like those facing the tobacco and food industries, this has two elements. The first concerns truth in advertising. Creams and cosmetics are making increasingly extravagant marketing claims. So far, women have been willing to buy into the illusion. Should that change (and there are signs it might), then manufacturers expose themselves to potentially ruinous litigation.

Second, there is a moral dimension. The beauty industry is at a stage where it can permanently change a person's looks. Given advances in genetic engineering and the competitive drive, a race for beauty is conceivable in which people will strive to model themselves on some form of idealised human being. By selling the weapons to win this war, the industry may find itself roundly condemned and subject to legislation.

Public handwringing is already evident in the case of teenagers indulging in cosmetic surgery. In “Branded”, a book on marketing to teenagers, Alissa Quart notes that in America the number of teenage breast implants and liposuctions rose by 562% between 1994 and 2001. There is a cynical marketing phrase for all this: helping “kids look older younger”. A number of new books have begun to question the ethics of marketing beauty products and services to adults too.

Part of the backlash so far is tighter regulation. Europe recently passed new labelling and animal-testing laws on cosmetics and will soon give the public the right to probe how firms create cosmetics. This is sure to raise costs for the industry worldwide. Much of the concern, however, is misdirected. Compared with the lead or belladonna that once gave women gothic dreamy eyes, today's lotions are relatively harmless, and consumers are well informed. Cosmetic-surgery techniques are improving—liposuction using electrically vibrating rods, not manual jabbing, still sounds horrible, but it is safer. Worries about newish procedures such as botox look overdone too—the stuff has been around for a decade as a medical treatment. Even silicone breast implants may be back on the market soon, after improvements in technology.

The fact is that neither moral censure nor fears about safety will stop people from wanting to look better. The desire is too entrenched. An 18th-century British law proposing to allow husbands to annul marriages to wives who had trapped them with “scents, paints, artificial teeth, false hair and iron stays”, had no effect on women, who continued to clamour for the latest French skin creams. During the second world war, the American government had to reverse a decision to remove lipstick from its list of essential commodities in order to prevent a rebellion by female war workers. The beauty business—the selling of “hope in a jar”, as Charles Revson, the founder of Revlon, once called it—is as permanent as its effects are ephemeral.