Early in 1895, while facing charges of indecency and wondering if he should abscond to France, Oscar Wilde had no idea what a two-year prison sentence would mean for him. Wilde’s misfortune was to serve his sentence just before prison conditions were officially changed by the 1898 Prison Act.

In total isolation, first in Pentonville and Wandsworth, and then in Reading gaol, to which he was moved in November 1895, Wilde slept on a plank bed with no mattress. Allowed one hour’s exercise a day, he walked in single file in the yard with other prisoners but he was not allowed to communicate with them. He could not sleep, he was permanently hungry and he suffered from dysentery. For the first month, Wilde was tied to a treadmill six hours a day, making an ascent, as it were, of 6,000 feet each day, with five minutes’ rest after every 20 minutes. Towards the end of his sentence, the governor of the jail, Major Nelson, remarked to Wilde’s friend Robert Ross: “He looks well. But like all men unused to manual labour who receive a sentence of this kind, he will be dead within two years.” Wilde was later to praise Nelson, who had arrived at Reading in July 1896, as “the most Christlike man I ever met”. Nelson was more liberal than his predecessor and was ready to relax the rules.

In January 1897, when Wilde still had more than four months still to serve, he and Nelson came up with an ingenious idea. While nothing in the prison regulations allowed prisoners to write plays or novels or essays, inmates had permission to write letters. Under the previous regime, Wilde had written to solicitors and the Home Office, or in limited quantities to friends, but his letters were inspected and the writing materials removed as he finished. But the regulations did not specify how long a letter should be. And if a letter were not finished, then the prisoner, it was supposed, could be allowed take it with him when he left the prison.

Thus Wilde, alone in his cell, was given pen and ink every day. What he wrote was removed each evening and then, it seems, handed back to him in the morning, or parts were given back to him to revise. Since “De Profundis” was in the form of a long letter, it would be his property when he was free. It took him three months, with much revision.

Facebook Twitter Pinterest Oscar Wilde in around 1890. Photograph: Print Collector/Getty Images

Wilde addressed his letter to Lord Alfred Douglas. On his release, he handed the manuscript to Ross, who had two typed copies made, one of which he sent to Douglas. In 1905 Ross published extracts from the text, and a fuller version in 1908. The complete version, however, was not published until 1949.

By the time he wrote “De Profundis”, Wilde’s love for Douglas had turned into a sort of bitterness, and the tone of his long letter manages to capture that bitterness as well as the extraordinary attachment he felt for Douglas. “De Profundis”, it should be said, is neither fair minded nor consistent; it is, at times, bloated in its comparisons and its rhetoric. (Wilde, for example, compares himself to Christ.) But there is also a beautiful, calm eloquence, and a sense of urgency, of things being said because there might not be time or opportunity to say them in the future.

Wilde’s old skill at paradox and phrase making was not there now merely to amuse his audience or mock his betters, but rather to kill his own pain and grief and attempt to communicate passionately and fiercely with his lover. He wrote not as art now, but as desperately serious matter.

“The only beautiful things,” his character Vivian had told us in “The Decay of Lying”, which Wilde wrote in 1890, “are the things that do not concern us.” Seven years later, in his cry from the depths, he wrote of what most deeply concerned him: “If there be one single passage that brings tears to the eyes, weep as we weep in prison when the day no less than the night is set apart for tears.” And then, in what is perhaps the most shocking sentence in the whole letter, he wrote: “The supreme vice is shallowness.” Once upon a time it would have been, for him, the supreme virtue.

Wilde accused Douglas of distracting him from his art, of spending his money, of degrading him ethically, of constant scene-making, of deliberately and then thoughtlessly mistreating him. He went over Douglas’s bad behaviour in matters large and petty, often citing dates and places and details. The tone was fluent and sweeping, full of carefully controlled cadence and measured elegance. But the difference between this tone and Wilde’s carefully nurtured flippancy was astonishing. It was like a tenor becoming a baritone, with a new range and depth and a new attention to feeling, but the old skills and tricks with pitch and paradox still in place, despite his circumstances, or perhaps because of them.

The letter cannot be read for its accurate account of their relationship, nor taken at its word. While some of the accusations are true, others are petty and foolish. But that is not the point. “De Profundis” has neither the informality of a personal letter nor the distilled sound of a piece of imaginative writing. Its seductive, hurt and passionate tone places it in a category of its own. In all its urgency and ambiguous eloquence, it remains one of the greatest and most complex love letters ever written.

It is fascinating to witness the change that came over Wilde’s imaginative procedures as he wrote. In the dim light of the prison cell and with the memory of his suffering fresh, it was as though he sought a new sort of tension between breathlessness and breath-control. He wrote long and highly wrought sentences, loving lists of adjectives and clever, Latinate diction and elaborate punctuation. To be followed by a pure, plain statement, full of the clipped sharp tone of the Anglo-Saxon.

“Of course I should have got rid of you,” he wrote. The reader will want to know why he did not, and part of the power of the text comes from our knowledge that, once Wilde was released, despite all his bitter feeling, he and Douglas attempted to live together again and resume their relationship, to the horror of both Douglas’s family and Wilde’s wife.

Facebook Twitter Pinterest Colm Tóibín in the cell of Reading prison where De Profundis was written.

Until three years ago, Reading gaol, where Wilde served most of his sentence, was used as a place of incarceration for young offenders. The cells still have bunk beds, a small table and two seats riveted into the wall, and then, closer to the small window, a flush toilet hidden from view. The four corridors that make up each floor radiate from a central vantage point. Thus it is possible for one single member of staff to note any movement in the four corridors.

While the creepy cell interiors bear all the hallmarks of recent use, the corridors and the general atmosphere in the prison, which was built in 1844, make it easy to imagine what it might have been like for Wilde in 1895.

The place where floggings happened, for example, which Wilde evoked with such passionate shock in his letters to the Daily Chronicle on his release, is close to the yard where the prisoners could exercise, close enough for the victim’s howls to be heard, as Wilde heard them in 1897. The cells that were used to house those condemned to death, evoked by Wilde in The Ballad of Reading Gaol, can still be visited. They are still part of the essential fabric of the place.

Wilde was not prepared for prison, and did not, in his heyday, dream that he would spend two years in jail. In “The Decay of Lying”, he had mocked the writer Charles Reade for “a foolish attempt to be modern, to draw public opinion to the state of our convict prisons”. In 1882, during a tour of America, when he visited a penitentiary in Nebraska, he wrote about the prisoners: “They were all mean looking, which consoled me, for I should hate to see a criminal with a noble face.”

When he found that a prisoner due to be hanged in Nebraska was reading a novel by Charlotte M Yonge called The Heir of Redclyffe, Wilde remarked: “My heart was turned by the eyes of the doomed man, but if he reads The Heir of Redclyffe it’s perhaps as well to let the law take its course.” But when he found a volume by Dante in another cell, he wrote: “Strange and beautiful it seemed to me that the sorrow of a single Florentine in exile should, hundreds of years afterwards, lighten the sorrow of some common prisoner in a modern gaol.”

The cell where Wilde wrote “De Profundis” is filled with a sense of those who recently suffered there, but the view of the sky from the small window and the heaviness of the door as it closes behind you bring back how one of the great spirits of his age was broken in body and soul in this space, and then restored to some dark approximation of life by being given pen and paper so that he too could lighten his sorrow, or make it matter more, by producing something as strange and beautiful as “De Profundis”.

• An exhibition, Inside – Artists and Writers in Reading Prison, is at the prison from 4 September to 30 October. Colm Tóibín will read “De Profundis” in its entirety on 16 October. artangel.org.uk/inside.