Some people only become famous after they die. When Dr James Barry died in 1865 he became infamous. By all accounts, he had led a colourful life. A renowned military surgeon, he rose to become inspector general of hospitals – one of the highest army medical posts – and served throughout the British empire. Notoriously irascible, Barry fought a duel with a fellow officer, ticked off Florence Nightingale and survived several army inquiries into his conduct. He was a humane doctor, fervent public health reformer and famous for his peculiarities: a teetotaller and vegetarian, he travelled with a menagerie of small animals.

Yet all these eccentricities were as nothing compared with the revelations that emerged on Barry’s death. For the brilliant Dr Barry was, in fact, a woman. The charwoman who washed the body discovered “he” was “a perfect female” and furthermore surmised – from stretch marks on the abdomen – that she had once given birth. More than 50 years before women were allowed to practise medicine, Barry had hoodwinked Edinburgh University, the Royal College of Surgeons and the British Army to become the first female doctor in the UK. At a time when women were barred from most formal education and most professions, she had masqueraded as a man in a life-long deception of breathtaking proportions.

The scandal that bewitched Victorian society spawned an army of commentators vying to determine who the real Dr Barry was and what led him/her to live a life of such exquisite subterfuge. Newspapers speculated that she was the illegitimate daughter of George III. Charles Dickens raked over the story in an article entitled “A Mystery Still”. And as that mystery deepened, Dr Barry’s tantalising tale was explored in novels and even a play, staged in 1919, with Sybil Thorndike in the starring role; after almost killing his duelling opponent, Thorndike’s Barry burst into feminine tears.

The true story is both more prosaic and infinitely more strange. For as the first biography, by Isobel Rae in 1958, speculated and later biographies agreed, the pistol-toting, foul-mouthed Dr Barry was really the daughter of a feckless Irish shopkeeper.

This latest biography is the longest and most comprehensive dissection of Barry’s life – and curious anatomy – so far. Retired surgeon Michael du Preez has devoted a decade to an archival detective trail, and together with biographer and novelist Jeremy Dronfield has shaped a scintillating portrait of Barry’s life. Tracking down new documentary evidence, they have produced irrefutable proof that Barry was born Margaret Bulkley in Cork in about 1790. Thanks to dogged sleuthing, the book fleshes out her early life, pinpoints the date of her dramatic transformation and follows Barry’s career across the globe.

A bright and spirited girl, 18-year-old Margaret chastised her spendthrift brother with the words: “Were I not a girl, I would be a soldier!” The following year she set out on the path towards that unlikely ambition when she exchanged her skirts for breeches and enrolled as a medical student at Edinburgh, shrewdly reducing her age to explain her hairless chin and petite frame. Helped by a small fortune from her late uncle, the artist James Barry, she not only adopted his name and presented herself as his nephew but acquired some of his influential patrons.

The diminutive Barry was adept at making powerful friends, who frequently got him out of scrapes. Lord Buchan intervened when the university declared him too young to sit his degree examination; he obtained a degree in medicine aged 22 when the authorities suspected he was only 12. After passing the Royal College of Surgeons examination, Barry signed up in the army as an assistant surgeon – though Buchan had to intervene again when his first commanding officer thought him only a boy.

Few truly suspected Barry's identity in his lifetime, though it seems likely some close allies shared his secret

Embarked on a glittering army career, Barry served 12 years in the Cape colony – eventually taking charge of all military medical matters there – before travelling to the West Indies, Mediterranean and Canada. With the aid of stacked heels and strategically placed stuffing, Barry even acquired a reputation as a “ladykiller”. But the tiny Dr Barry also made dangerous enemies. Perhaps overcompensating for his feminine appearance – or to keep over-friendly admirers at bay – he was frequently bad-tempered, coarse and even violent. At the Cape, Barry was renowned for shouting at patients, arguing with superiors and throwing medicine bottles at the wall – though luckily, when he challenged an army captain to a duel, both escaped without serious injury. Florence Nightingale felt the lash of his tongue at the Scutari barracks, in Istanbul, when Barry scolded her for wearing only a cap in the sun. He courted scandal, too. An inquiry was launched into a poster that accused Lord Charles Somerset, the Cape governor, of “buggering Dr Barry”. The accuser was not identified and the allegations never substantiated but Du Preez and Dronfield suspect the pair may indeed have had an affair and that Somerset – 18 years senior – knew he was a woman. Barry would later describe Somerset as “my more than father – my almost only friend”. Yet Barry was also a skilled surgeon, who performed one of the first successful caesareans, and a committed sanitary reformer. He lambasted the authorities for mismanagement of barracks, prisons and asylums, while treating rich and poor, colonists and slaves alike.

Facebook Twitter Pinterest Dr James Barry. Photograph: Oneworld Publications

Tracing Barry’s life against the backdrop of the British empire, Du Preez and Dronfield tell a story that feels almost Dickensian in style, and – like Dickens – it is sometimes long-winded and overblown. The vignettes that open some chapters, imagining scenes in which Barry gazes out to sea or stands on deck, undermine the authenticity of the admirable research.

At least the central question at the heart of Dr Barry’s mystery – who? – can now be answered with near certainty. Yet the how and why remain tantalisingly veiled. On Barry’s death, acquaintances queued up to assert they had always guessed he was a woman, while some even claimed to have seen the proof. Few truly suspected his identity in his lifetime, though it seems likely some close allies shared his secret. In her 2002 biography, Rachel Holmes made a plausible case for Margaret being intersex: born with male and female characteristics, she was brought up female then opted to live as a man. Barry’s Edinburgh thesis on femoral hernias (hernias of the thigh) – which can turn out to be descended testicles – gives weight to this idea. Du Preez and Dronfield dismiss this notion and argue Barry’s decision to live as a man was “motivated more by ambition than identity”.

The doctor who signed Barry’s death certificate said it was “none of my business” whether Barry was male or female – and perhaps he was right. The truth remains buried in a grave in Kensal Green cemetery, north-west London. All that can be said with certainty is that the enigma surrounding Barry is set to continue.

• To order Dr James Barry (published by Oneworld) for £13.93 (RRP £16.99) go to bookshop.theguardian.com or call 0330 333 6846. Free UK p&p over £10, online orders only. Phone orders min p&p of £1.99.