Carl Djerassi can remember the moment when he became a writer. It was 1993, he was a professor of chemistry at Stanford University in California and he had already written books about science and about his life as one of the inventors of the Pill. Now he wanted to write a literary novel about writers’ insecurities, with a central character loosely modelled on Norman Mailer, Philip Roth and Gore Vidal.

His wife, Diane Middlebrook, thought it was a ridiculous idea. She was also a professor—of literature. “She admired the fact that I was a scientist who also wrote,” Djerassi says. He remembers her telling him, “‘You’ve been writing about a world that writers know little about. You’re writing the real truth inside of almost a closed tribe. But there are tens of thousands, hundreds of thousands of people who know more about writing than you do. I advise you not to do this.’”

Even at 85, slight and snowy-haired, Djerassi is a det­ermined man. You sense his need to prove that he can, he will prevail. Sitting in his London flat, he leans forward to fix me with his hazel eyes. “I said, ‘ok. I’m not going to show it to you till I finish. And if I find a publisher then I’ll give it to you.’ ”

Eventually Djerassi got the bound galleys of his book. “We were leaving San Francisco for London for our usual summer and I said ‘Look, would you read this now?’ She said, ‘Sure, on the plane.’ So my wife sits next to me and of course I sit and look over. And I still remember, I had a Trollope, 700 pages long, and I couldn’t read anything because I wanted to see her expression.”

Diane Middlebrook died of cancer in 2007 and, as Djerassi speaks, her presence grows stronger. By the end it is as if there are three of us in the room. “She was always a fantastic reader,” he says. “She read fast and continuously. And suddenly you hear the snap of the book closing, like a thunder clap. And I looked at her, and she then looked at me. She always used to call me, not ‘Carl’ or ‘Darling’, she used to call me ‘Chemist’ in a dear, affectionate sort of way. It was always ‘Chemist’. And she said, ‘Chemist, this is good’.”

Carl Djerassi is a polymath. Strictly speaking that means he is someone who knows a lot about a lot. But Djerassi also passes a sterner test: he can do a lot, too. As a chemist (synthesising cortisone and helping invent the Pill); an art collector (he assembled one of the world’s largest collections of works by Paul Klee); and an author (19 books and plays), he has accomplished more than enough for one lifetime.

His latest book, “Four Jews on Parnassus”, is an ima­gined series of debates between Theodor Adorno, Arnold Schönberg, Walter Benjamin and Gershom Scholem, which touches on art, music, philosophy and Jewish identity. In itself, the book is an exercise in polymathy. At a reading in the Austrian Cultural Forum in London this summer, complete with Schönberg’s songs and four actors, including Djerassi himself, it drew a good crowd and bewitched them for an hour and a half. Sitting down with the book the next day, I found it sharp, funny, mannered and dazzlingly erudite—sometimes, like a bumptious student, too erudite for its own good. I enjoy Djerassi’s writing, though not everyone will. But even his critics would admit that he really is more than “a scientist who writes”.

The word “polymath” teeters somewhere between Leo­nardo da Vinci and Stephen Fry. Embracing both one of history’s great intellects and a brainy actor, writer, director and TV personality, it is at once presumptuous and banal. Djerassi doesn’t want much to do with it. “Nowadays people that are called polymaths are dabblers—are dabblers in many different areas,” he says. “I aspire to be an intellectual polygamist. And I deliberately use that metaphor to provoke with its sexual allusion and to point out the real difference to me between polygamy and promiscuity."

“To me, promiscuity is a way of flitting around. Polygamy, serious polygamy, is where you have various marriages and each of them is important. And in the ideal polygamy I suspect there’s no number one wife and no number six wife. You have a deep connection with each person.”

Djerassi is right to be suspicious of flitting. We all know a gifted person who cannot stick at anything. In his book “Casanova: A Study in Self-Portraiture” Stefan Zweig describes an extreme case:

[Casanova] excelled in mathematics no less than in philosophy. He was a competent theologian, preaching his first sermon in a Venetian church when he was not yet 16 years old. As a violinist, he earned his daily bread for a whole year in the San Samuele theatre. When he was 18 he became doctor of laws at the University of Padua—though down to the present day the Casanovists are still disputing whether the degree was genuine or spurious...He was well informed in chemistry, medicine, history, philosophy, literature, and, above all, in the more lucrative (because perplexing) sciences of astrology and alchemy...As universal dilettante, indeed, he was perfect, knowing an incredible amount of all the arts and all the sciences; but he lacked one thing, and this lack made it impossible for him to become truly productive. He lacked will, resolution, patience.

Mindful of that sort of promiscuity, I asked my colleagues to suggest living polymaths of the polygamous sort—doers, not dabblers. One test I imposed was breadth. A scientist who composes operas and writes novels is more of a polymath than a novelist who can turn out a play or a painter who can sculpt. For Djerassi, influence is essential too. “It means that your polymath activities have passed a certain quality control that is exerted within each field by the competition. If they accept you at their level, then I think you have reached that state rather than just dabbling.” They mentioned a score of names—Djerassi was prominent among them. Others included Jared Diamond, Noam Chomsky, Umberto Eco, Brian Eno, Michael Frayn and Oliver Sacks.

It is an impressive list, by anyone’s standards. You can find scientists, writers, actors, artists—the whole range of human creativity. Even so, what struck me most strongly was how poorly today’s polymaths compare with the polymaths of the past.

In the first half of 1802 a physician and scientist called Thomas Young gave a series of 50 lectures at London’s new Royal Institution, arranged into subjects like “Mechanics” and “Hydro­dynamics”. By the end, says Young’s biographer Andrew Robinson, he had pretty much laid out the sum of scientific knowledge. Robinson called his book “The Last Man Who Knew Everything".

Young’s achievements are staggering. He smashed Newtonian orthodoxy by showing that light is a wave, not just a particle; he described how the eye can vary its focus; and he proposed the three-colour theory of vision. In materials science, engineers dealing with elasticity still talk about Young’s modulus; in linguistics, Young studied the grammar and voc­abulary of 400 or so languages and coined the term “Indo-European”; in Egyptology, Jean-François Champollion drew on his work to decode the Rosetta stone. Young even tinkered around with life insurance.

When Young was alive the world contained about a billion people. Few of them were literate and fewer still had the chance to experiment on the nature of light or to examine the Rosetta stone. Today the planet teems with 6.7 billion minds. Never have so many been taught to read and write and think, and then been free to choose what they would do with their lives. The electronic age has broken the shackles of knowledge. Never has it been easier to find something out, or to get someone to explain it to you.

Yet as human learning has flowered, the man or woman who does great things in many fields has become a rare species. Young was hardly Aristotle, but his capacity to do important work in such a range of fields startled his contemporaries and today seems quite bewildering. The dead cast a large shadow but, even allowing for that, the 21st century has no one to match Michelangelo, who was a poet as well as a sculptor, an architect and a painter. It has no Alexander von Humboldt, who towered over early-19th-century geography and science. And no Leibniz, who invented calculus at the same time as Newton and also wrote on technology, philosophy, biology, politics and just about everything else.

Although you may be able to think of a few living polymaths who rival the breadth of Young’s knowledge, not one of them beg­ins to rival the breadth of his achievements. Over the past 200 years the nature of intellectual endeavour has changed profoundly. The polymaths of old were one-brain universities. These days you count as a polymath if you excel at one thing and go on to write a decent book about another.

Young was just 29 when he gave his lectures at the Royal Institution. Back in the early 19th century you could grasp a field with a little reading and a ready wit. But the distinction between the dabbling and doing is more demanding these days, because breaking new ground is so much harder. There is so much further to trek through other researchers’ territory before you can find a patch of unploughed earth of your own.

Even the best scientists have to make that journey. Benjamin Jones, of the Kellogg School of Management near Chicago, looked at the careers of Nobel laureates. Slightly under half of them did their path-breaking work in their 30s, a smattering in their 20s—Einstein, at 26, was unusually precocious. Yet when the laureates of 1998 did their seminal research, they were typically six years older than the laureates of 1873 had been. It was the same with great inventors.

Once you have reached the vanguard, you have to work harder to stay there, especially in the sciences. So many scientists are publishing research in each specialism that merely to keep up with the reading is a full-time job. “The frontier of knowledge is getting longer,” says Professor Martin Rees, the president of the Royal Society, where Young was a leading light for over three decades. “It is impossible now for anyone to focus on more than one part at a time.”

Specialisation is hard on polymaths. Every moment devoted to one area is a moment less to give over to something else. Researchers are focused on narrower areas of work. In the sciences this means that you often need to put together a team to do anything useful. Most scientific papers have more than one author; papers in some disciplines have 20 or 30. Only a fool sets out to cure cancer, Rees says. You need to concentrate on some detail—while remembering the big question you are ultimately trying to answer. “These days”, he says, “no scientist makes a unique contribution.”

It is not only the explosion of knowledge that puts polymaths at a disadvantage, but also the vast increase in the number of specialists and experts in every field. This is because the learning that creates would-be polymaths creates monomaths too and in overwhelming numbers. If you have a multitude who give their lives to a specialism, their combined knowledge will drown out even a gifted generalist. And while the polymath tries to take possession of a second expertise in some distant discipline, his or her first expertise is being colonised by someone else.

The arts are more forgiving than the sciences. Rees is reminded of a remark by Peter Medawar, the zoologist, who pointed out that, after finishing a draft of “Siegfried” in 1857, Wagner was able to put the opera aside for 12 years before setting out to complete his Ring Cycle with “Götterdämmerung”. A scientist would have had to worry about a rival stealing his thunder. But nobody else was about to compose the destruction of Valhalla.

Perhaps that explains why would-be polymaths these days so often turn to writing books. Yet, as Richard Posner has discovered, even that is often enemy territory.

Unlike France, America and Britain don’t tend to encourage public intellectuals. But if they did, Richard Posner would be their standard-bearer. Posner’s day job is as an appeals-court judge in Chicago—a career founded upon his reputation as America’s pre-eminent thinker on anti-trust law. But Posner is not just a lawyer. In his spare time he has written on sex, security, politics, Hegel, Homeric society, medieval Iceland and a whole lot more. The Wall Street Journal once called him a “one-man think-tank”.

Posner thinks like a polymath. “I’m impatient and I’m restless,” he says, in a matter-of-fact way. “After I graduated from law school, I worked first in government for six years. I enjoyed it but I didn’t really want to make a career of that. I went into teaching without any great sense of commitment, but I couldn’t think of anything else. But gradually I lost int­erest, as the 1970s wore on, I became involved in consulting. So when the judgeship came along in 1981—quite out of the blue—I was happy to take that. I just kind of slid into law. It is sort of the default career choice in the United States.”

Posner first made his name as a monomath. “I had a very big intellectual commitment for many years to anti-trust law. I wrote a lot about that.” Eventually, though, the polymath rose to the surface and he put anti-trust behind him. “I just got bored with it, I think the field slowed down—it happens with fields,” he says. These days most people cling to their expertise; Posner talks about it as if he were trading in an old car.

After he became immersed in the intellectual life of the University of Chicago, Posner started to apply insights from economics to a broad range of subjects. In his book “Sex and Reason”, written in 1990, he used economics to explain a part of life that specialist lawyers and economists had tended to think was beyond their reach. To take a simple example, the AIDS epidemic made gay sex unavoidably more costly, either because of the risk of disease or of switching to safe sex. It therefore reduced the amount of gay sex—and, by the same mechanism, cut the number of illegitimate births and inc­reased the number of legitimate ones.

The book was a success because Posner had the field pretty much to himself. “Sometimes one goes into a new area and there hasn’t been much done in it and then you are a little ahead of the curve,” he says. Even then, the monomaths were in hot pursuit. “After a while there is so much in it that you don’t know what you’re going to do. Since 1990 the field has become extremely crowded because of specialisation and not very attractive.” Time to move on.

The monomaths do not only swarm over a specialism, they also play dirty. In each new area that Posner picks—policy or science—the experts start to erect barricades. “Even in relatively soft fields, specialists tend to develop a specialised vocabulary which creates barriers to entry,” Posner says with his economic hat pulled down over his head. “Specialists want to fend off the generalists. They may also want to convince themselves that what they are doing is really very difficult and challenging. One of the ways they do that is to develop what they regard a rigorous methodology—often mathematical.

“The specialist will always be able to nail the generalists by pointing out that they don’t use the vocabulary quite right and they make mistakes that an insider would never make. It’s a defence mechanism. They don’t like people invading their turf, especially outsiders criticising insiders. So if I make mistakes about this economic situation, it doesn’t really bother me tremendously. It’s not my field. I can make mistakes. On the other hand for me to be criticising someone whose whole career is committed to a particular outlook and method and so on, that is very painful.”

For a polymath, the charge of dabbling never lies far below the surface. “With the amount of information that’s around, if you really want to understand your topic thoroughly then, yes, you have to specialise,” says Chris Leek, the chairman of British Mensa, a club for people who score well on IQ tests. “And if you want to speak with authority, then it’s important to be seen to specialise.”

That is why modern institutions tend to exclude polymaths, he says. “It’s very hard to show yourself as a polymath in the current academic climate. If you’ve got someone interested in going across departments, spending part of the time in physics and part of the time elsewhere, their colleagues are going to kick them out. They’re not contributing fully to any single department. OK, every so often you’re going to get a huge benefit, but from day to day, where the universities are making appointments, they want the focus in one field.”

Britain goes out of its way to create monomaths, by asking students aged 15 to choose just three or four subjects to study at A-level. Djerassi thinks this is a mistake. “There’ll be students here at age 16 or 17 who are much better than many Americans at French or maths or something, but abysmally ignorant in another area,” he says. “We really preach intellectual monogamy more and more in this day and age. That’s by necessity, but we’re overdoing it. And what we really ought to do is start with intellectual polygamy.”

Djerassi has also suffered in his own work because of monomaths’ hostility, especially as a playwright. “They always keep crying out ‘the co-inventor, father, the mother of the Pill’,” he growls. “Without having any knowledge about the play, they start with it. As if it’s got anything to do with it.” Djerassi thinks that this means he has to work harder to promote his work. “No agent has ever been interested in me. They want 29-year-old Irish playwrights, not 86-year-old expatriates.” A trace of bitterness creeps into his voice, but he concedes: “If I were an agent I’d feel the same way.”

Overwhelmed by specialists and attacked by experts as dilettantes, it is amazing that there are any polymaths at all. How do they manage?

Alexander McCall Smith is a natural writer. “I just have to do it,” he says. “I suppose I write four novels a year now, which I don’t have to do. In one sense, that is breaking all the rules in publishing: you’re only meant to write one, but I write four, sometimes five. But I just feel that I have got to do it and I enjoy it greatly. I suppose I am very fortunate. The way I work is I go into a trance and write. I don’t have to sit there and think: it happens. It just comes, so I am very, very lucky.”

These days McCall Smith is best-known as the man behind “The No. 1 Ladies’ Detective Agency”. But his first career, as a university professor, was eminent in its own right. “My interest was medical law. That, I suppose, was cross-disciplinary. You had to be able to understand the scientific issues and the medical issues, but you just had to have a sound lay understanding of them. So, for example, I worked as a member of the Human Genetics Commission for a while. And that meant I had to go off and make sure that I understood what the issues in genetics were.”

He is also musical—though in a dabbling way. “I play wind instruments, but I don’t play them very well,” he says. “My wife and I set up an orchestra, which is called the Really Terrible Orchestra, and indeed that is absolutely accurate. Virtually everybody I know is better at music than I am.”

McCall Smith is a polymath by necessity. He wrote while he was an academic, producing fiction, about 30 children’s books, short stories and plays for radio. He paid a price. “I probably would have made more of my academic career had I not had another interest, I think, yes. Academia requires a lot of commitment, so I suppose I could have done more.” But, speaking to him, I don’t think he had a choice.

Circumstance also played its part. McCall Smith was able to write because university life allowed it. “It would have been different had I been somebody who practised commercial law in a law firm, for instance. That wouldn’t be compatible with doing anything else. If you were a futures trader or something like that—there are some jobs where the pressure is so intense that it must be very difficult to have any energy by the time you come home at night.”

Posner could become a polymath because he has a unifying set of ideas. “A lot of this work is economic theory in new areas. So applying a method to a new field is not the same thing as mastering multiple fields. To achieve mastery in unrelated areas in an age of specialisation is exceedingly difficult. On the other hand, to take a technique that can be applied to a variety of substantive fields is not as difficult. So if I write about the economics of old age and the economics of sex and the economics of the national security and intelligence services, I am not mastering the field. I am not becoming a sociologist, or a psychiatrist or what have you.”

Djerassi could become a polymath because he has had two careers, one after the other—he did his science and, having made a fortune, he concentrated on his writing. He was helped by his wife. “She was a very sophisticated writer and an extremely tough critic and she managed to divorce affection from criticism. She thought ‘this is terrible’ or ‘this is clichéd’.” He also has ambition and the willpower of someone on borrowed time. At 62 he was diagnosed with cancer. “Suddenly, from one day to another, I didn’t even know what my life expectancy would be before I got the pathology back after the operation. And I remember being very depressed and afterwards I didn’t want to talk to anyone.” He said to himself, “‘Gee, now if I’d known five years earlier it would come out that I’d have cancer and be told I’d live for another few years, would I live a different life?’ And I said, ‘Absolutely’.”

Not all polymaths find their way. Andrew Robinson, Young’s biographer, gives the example of Michael Ventris, who died aged 34, having tried to satisfy both his urge to be an architect and also his fascination with codes. Ventris was the first to make sense of Linear B, an early Greek script, but he could not apply himself as successfully to architecture.

“With Michael Ventris, the polymathy gradually des­troyed him,” Robinson says. “He was famous for cracking Linear B, but I believe he was depressed. Architecture was not enough. He was a logician. Linear B took him over. He couldn’t reach the standard he had set in another field, he couldn’t do justice to his own gifts, he couldn’t let it all go and give it up.”

Robinson thinks that Young also ran up against his limits. “Young understood after 1814 that he couldn’t carry on with serious medicine. He could have pursued it but even then it was clear that he wouldn’t be taken seriously. People love a sole genius with tunnel vision—a focus,” Robinson says. Darwin spent several years thinking about barnacles. But because Young’s work was in so many different fields, he was accused of being a dilettante. “Polymaths are disconcerting,” Robinson says. “People feel they are trespassing.”

Even Leonardo warned against being spread thin. The other day Robinson came across one of his late notebooks, in which he had written, “Like a kingdom divided, which rushes to its doom, the mind that engages in subjects of too great variety becomes confused and weakened.”

In an age of specialists, does it matter that generalists no longer thrive? The world is hardly short of knowledge. Countless books are written, canvases painted and songs recorded. A torrent of research is pouring out. A new orthodoxy, popularised by Malcolm Gladwell, sees obsessive focus as the key that unlocks genius.

Just knowing about a lot of things has never been easier. Never before have dabblers been so free to paddle along the shore and dip into the first rock pool that catches the eye. If you have an urge to take off your shoes and test the water, countless specialists are ready to hold your hand.

And yet you will never get very deep. Depth is for monomaths—which is why experts so often seem to miss what really matters. Specialisation has made the study of English so sterile that students lose much of the joy in reading great literature for its own sake. A generation of mathematically inclined economists neglected many of Keynes’s insights about the Depression because he put them into words. For decades economists sweated over fiendish mathematical equations, only to be brought down to earth by the credit crunch: Keynes’s well-turned phrases had come back to life.

Part of my regret at the scarcity of polymaths is sentimental. Polymaths were the product of a particular time, when great learning was a mark of distinction and few people had money and leisure. Their moment has passed, like great houses or the horse-drawn carriage. The world may well be a better place for the specialisation that has come along instead. The pity is that progress has to come at a price. Civilisation has put up fences that people can no longer leap across; a certain type of mind is worth less. The choices modern life imposes are duller, more cramped.

The question is whether their loss has affected the course of human thought. Polymaths possess something that monomaths do not. Time and again, innovations come from a fresh eye or from another discipline. Most scientists devote their careers to solving the everyday problems in their specialism. Everyone knows what they are and it takes ingenuity and perseverance to crack them. But breakthroughs—the sort of idea that opens up whole sets of new problems—often come from other fields. The work in the early 20th century that showed how nerves work and, later, how DNA is structured originally came from a marriage of physics and biology. Today, Einstein’s old employer, the Institute for Advanced Study at Princeton, is laid out especially so that different disciplines rub shoulders. I suspect that it is a poor substitute.

Isaiah Berlin once divided thinkers into two types. Foxes, he wrote, know many things; whereas hedgehogs know one big thing. The foxes used to roam free across the hills. Today the hedgehogs rule.