History doesn't recount who gave Cesare Borgia syphilis, but we do know when and where he got it. In the summer of 1497, he was a 22-year-old cardinal, sent as papal legate by his father, Pope Alexander VI, to crown the king of Naples and broker a royal marriage for his sister, Lucrezia. Naples was a city rich in convents and brothels (a fertile juxtaposition in the male Renaissance imagination), but it was also ripe with disease. Two years earlier, a French invasion force including mercenary troops back from the new world, had dallied a while to enjoy their victory, and when they left, carried something unexpected and deadly back home with them.

His work accomplished, Cesare took to the streets. Machiavelli, his contemporary and a man with a wit as unflinching as his politics, has left a chilling account of his coupling with a prostitute who, when he lights a lamp afterwards, is revealed as a bald, toothless hag so hideous that he promptly throws up over her. Given Cesare's elevated status, his chosen women no doubt were more enticing, but the sickness they gave him (and suffered themselves) was to prove vicious. First a chancre appeared on his penis, then crippling pains throughout his body and a rash of itching, weeping pustules covering his face and torso. Fortunately for him and for history, his personal doctor, Gaspar Torella, was a medical scholar with a keen interest in this startling new disease and used his patient (under the pseudonym of "Niccolo the young") to record symptoms and attempted cures. Over the next few years, Torella and others charted the unstoppable rise of a disease that had grown men screaming in agony as their flesh was eaten away, in some cases down to the bone.

I still remember the moment, sitting in the British Library, when I came across details of Torella's treatise in a book of essays on syphilis. There is nothing more thrilling in writing historical fiction than when research opens a window on to a whole new landscape, and the story of how this sexual plague swept through Europe during the 1490s was one of the turning points in Blood and Beauty, the novel I was writing on the rise and fall of the Borgia dynasty.

By the time that Cesare felt that first itch, the French disease, as it was then known, had already spread deep into Europe. That same year, Edinburgh town council issued an edict closing brothels, while at the Italian university of Ferrara scholars convened an emergency debate to try to work out what had hit them. By then the method of the contagion was pretty obvious. "Men get it from doing it with women in their vulvas," wrote the Ferrarese court doctor baldly (there is no mention of homosexual transmission, but then "sodomy", as it was known then, was not the stuff of open debate). The theories surrounding the disease were are as dramatic as the symptoms: an astrological conjunction of the planets, the boils of Job, a punishment of a wrathful God disgusted by fornication or, as some suggested even then, an entirely new plague brought from the new world by the soldiers of Columbus and fermented in the loins of Neapolitan prostitutes.

Whatever the cause, the horror and the agony were indisputable. "So cruel, so distressing, so appalling that until now nothing more terrible or disgusting has ever been known on this earth," says the German humanist Joseph Grunpeck, who, when he fell victim, bemoaned how "the wound on my priapic gland became so swollen, that both hands could scarcely encircle it." Meanwhile, the artist Albrecht Dürer, later to use images of sufferers in propaganda woodcuts against the Catholic church, wrote "God save me from the French disease. I know of nothing of which I am so afraid … Nearly every man has it and it eats up so many that they die."

It got its name in the mid 16th century from a poem by a Renaissance scholar: its eponymous hero Syphilus, a shepherd, enrages the Sun God and is infected as punishment. Outside poetry, prostitution bears the brunt of the blame, though the real culprit was testosterone. Men infected prostitutes who then passed it on to the next client who gave it back to a new woman in a deadly spiral. Erring husbands gave it to wives who sometimes passed it on to children, though they might also get it from suckling infected wet-nurses.

Amid all this horror there were elements of poetic justice. In a manifestly corrupt church, the give-away "purple flowers" (as the repeated attacks were euphemistically known) that decorated the faces of priests, cardinals, even a pope, were indisputable evidence that celibacy was unenforceable. When Luther, a monk, married a nun, forcing the hand of the Catholic church to resist similar reform in itself, syphilis became one of the reasons the Catholic church is still in such trouble today.

Though there has been dispute in recent years over pre-15th-century European bones found with what resemble syphilitic symptoms, medical science is largely agreed that it was indeed a new disease brought back with the men who accompanied Columbus on his 1492 voyage to the Americas. In terms of germ warfare, it was a fitting weapon to match the devastation that measles and smallpox inflicted travelling the other way. It was not until 1905 that the cause of all this suffering was finally identified under the microscope – Treponema pallidum, a spirochete bacterium that enters the bloodstream and, if left untreated, attacks the nervous system, the heart, internal organs and the brain; and it was not until the 1940s and the arrival of penicillin that there was an effective cure.

Much of the extraordinary detail we now have about syphilis is a result of the Aids crisis. Just when we thought antibiotics, the pill and more liberal attitudes had taken the danger and shame out of sexual behaviour, the arrival out of nowhere of an incurable, fatal, highly contagious sexual disease challenged medical science, triggered a public-health crisis and re-awoke a moral panic.

Not surprisingly, it also made the history of syphilis extremely relevant again. The timing was powerful in another way too, as by the 1980s history itself was refocusing; from the long march of the political and the powerful, to the more intimate cultural stories of everyman/woman. The growth of areas such as history of medicine and madness through the work of historians such as Roy Porter and Michel Foucault was making the body a rich topic for academics. Suddenly, the study of syphilis became, well, there is no other word for it, sexy.

Historians mining the archives of prisons, hospitals and asylums now estimate that a fifth of the population might have been infected at any one time. London hospitals during the 18th century treated barely a fraction of the poor, and on discharge sufferers were publicly whipped to ram home the moral lesson.

Those who could buy care also bought silence – the confidentiality of the modern doctor/patient relationship has it roots in the treatment of syphilis. Not that it always helped. The old adage "a night with Venus; a lifetime with Mercury" reveals all manner of horrors, from men suffocating in overheated steam baths to quacks who peddled chocolate drinks laced with mercury so that infected husbands could treat their wives and families without them knowing. Even court fashion is part of the story, with pancake makeup and beauty spots as much a response to recurrent attacks of syphilis as survivors of smallpox.

And then there are the artists; poets, painters, philosophers, composers. Some wore their infection almost as a badge of pride: The Earl of Rochester, Casanova, Flaubert in his letters. In Voltaire's Candide, Pangloss can trace his chain of infection right back to a Jesuit novice who caught it from a woman who caught it from a sailor in the new world. Others were more secretive. Shame is a powerful censor in history, and in its later stages syphilis, known as the "great imitator", mimics so many other diseases that it's easy to hide the truth. Detective work by writers such as Deborah Hayden (The Pox: Genius, Madness, and the Mysteries of Syphilis) count Schubert, Schumann, Baudelaire, Maupassant, Flaubert, Van Gogh, Nietzsche, Wilde and Joyce with contentious evidence around Beethoven and Hitler. Her larger question – how might the disease itself have affected their creative process – is a tricky one.

Van Gogh paints skulls and Schubert's sublime last works are clearly suffused with the awareness of death. But in 1888, when Nietzsche, tumbling into insanity, wrote work such as Ecce Homo is his intellectual grandiosity genius or possibly the disease talking? There is a further layer of complexity to this. By the time Nietzsche lost his wits, tertiary syphilis had undergone a transmutation, infecting the brain and causing paralysis alongside mental disintegration. But many of its sufferers didn't know that then. Guy de Maupassant, who started triumphant ("I can screw street whores now and say to them 'I've got the pox.' They are afraid and I just laugh"), died 15 years later in an asylum howling like a dog and planting twigs as baby Maupassants in the garden.

Late 19th-century French culture was a particularly rich stew of sexual desire and fear. Upmarket Paris restaurants had private rooms where the clientele could enjoy more than food, and in opera foyers patrons could view and "reserve" young girls for later. At the same time, the authorities were rounding up, testing and treating prostitutes, often too late for themselves or the wives. As the fear grew, so did the interest in disturbed women. Charcot's clinic exhibited examples of hysteria, prompting the question now as to how far that diagnosis might have been covering up the workings of syphilis. Freud noted the impact of the disease inside the family when analysing his early female patients.

"It's just as I thought. I've got it for life," says the novelist Alphonse Daudet after a meeting with Charcot in 1880s. In his book In the Land of Pain, translated and edited by Julian Barnes in 2002, the writer's eye is unflinching as he faces "the torment of the Cross: violent wrenching of the hands, feet, knees, nerves stretched and pulled to breaking point," dimmed only by the blunt relief of increasing amounts of morphine: "Each injection [helps] for three or four hours. Then come 'the wasps' stinging, stabbing here, there, everywhere followed by Pain, that cruel guest … My anguish is great and I weep as I write."

Of course, we have not seen the end of syphilis – worldwide millions of people still contract it, and there are reports, especially within the sex industry, that it is on the increase in recent years. But the vast majority will be cured by antibiotics before it takes hold. They will never reach the point, as Cesare Borgia did in the early 16th century, of having to wear a mask to cover the ruin of what everyone agreed was once a most handsome face. What he lost in vanity he gained in sinister mystery. How far his behaviour, oscillating between lethargy and manic energy, was also the impact of the disease we will never know. He survived it long enough to be cut to pieces escaping from a Spanish prison. Meanwhile, in the city of Ferrara,his beloved sister Lucrezia, then married to a duke famed for extramarital philandering, suffered repeated miscarriages – a powerful sign of infection in female sufferers. For those of us wedded to turning history into fiction, the story of syphilis proves the cliche: truth is stranger than anyone could make up.

• A Cultural History of Syphilis will be broadcast on Radio 3 on 26 May.