Three years have passed since the collapse of the Tower of Pisa, but only now can I accept the crucial role that I played in the destruction of this unique landmark. Over twenty tourists died as the thousands of tons of marble lost their grasp on the air and collapsed to the ground. Among them was my wife Elaine, who had climbed to the topmost tier and was looking down at me when the first visible crack appeared in the tower's base. Never were tragedy and triumph so intimately joined, as if Elaine's pride in braving the worn and slippery stairs had been punished by the unseen forces that had sustained this unbalanced mass of masonry for so many centuries.

I realise now that another element - farce - was present on that day. By chance a passing tourist on the steps of the cathedral had taken a photograph of the tower as the crack reached the third floor and a tell-tale section of cornice began its fall to earth. The photograph, endlessly published throughout the world, clearly shows the four startled tourists on the uppermost deck. Three of them are leaning back on their heels, hands raised to grip the sky, aware that the ancient campanile has moved under their feet.

Elaine, alone, has already seized the rail, and is staring at the grass waiting for her nearly two hundred feet below. Using a magnifying glass, one can see that, true to her quirky and mocking character, she shows almost no alarm. Her eyes have noticed the falling cornice, and I like to think that she is already planning to sue the municipality of Pisa for neglecting the safety of its tourists, and is collecting evidence that in due course she will present to her lawyers.

The dozen or so tourists visible on the lower floors are still making their way around its canted decks, groping past the narrow columns as they climb the 300 steps to the roof. A father and his young daughter wave to the tourists below them, two Italian sailors in uniform play the fool for their girl-friends, feigning an attack of giddiness, and an elderly couple pause to rest after climbing to the first floor, determined to complete the ascent. None of them sees the falling cornice and the fine cascade of powdered mortar.

The only figure on the ground who is aware of the imminent catastrophe is a man in a white jacket and panama hat who stands at the foot of the tower, both hands raised to the marble flank. His face is hidden, but his arms are braced against the shifting stone, his back arched above his straining legs. We can see that in his desperate way he is trying to hold upright the collapsing tower that is about to obliterate him.

Or so everyone assumes. The newspaper caption writers, the commentators on TV documentaries, all commend the bravery of this solitary figure. Surprisingly, he has never been identified, and neither his hat nor his white jacket were found in the mountain of rubble that was later removed, stone by stone, from the unhappy site.

But was he trying to support the tower or, rather, helping it on its way? I, of course, can answer the question, since I am the man in the panama hat, the husband at whom Elaine, in the last moments of her life, so triumphantly stares.

* * *

Needless to say, I fled to safety, running through the dust and the shrieking tourists as the ground trembled and a cataract of masonry fell from the air. A vast cloud of pulverised marble enveloped the square, and I remember stumbling past the horror-stricken waiters and taxi-drivers who gazed at this field of devastation - not only had the tower vanished, but it had taken their livelihoods with it. Had they known that I was responsible they would have lynched me on the spot, and to this day I have kept silent, still gripped by my guilt over so many deaths, all but one of them entirely innocent.

* * *

In a sense the destruction of the tower was inscribed days beforehand in our unhappy tour of Tuscany. Our marriage, problematic from the start, had grown increasingly fraught during the previous year. Elaine had married me on the rebound, to spite an unfaithful lover, but soon decided that her husband, a classics lecturer at a minor university, was minor in all other respects. I was losing my students in a ferment of curriculum changes that would eventually lead to the descheduling of Latin and Greek and their replacement by cultural and media studies. My refusal to sue the university, Elaine decided, was a sign of my innate weakness, a frailty that soon extended to the marriage bed.

Claiming that our union was unconsummated, she consulted a solicitor with a view to divorcing me, but was persuaded to make a last effort to save the relationship. Our marriage became a series of negotiated truces, in which I would yield more and more territory. Still hoping to salvage something, and return to the few weeks of happiness we had known after the wedding, I suggested a holiday in Italy. I had arranged to give three lectures at the University of Florence, which would pay for our air fares, and then we would be free to enjoy ourselves in the Tuscan countryside.

Elaine agreed, but only grudgingly - her first husband had been a modernist architect, and she always claimed to dislike the past, the territory I had made my own, and pretended to prefer California and Texas. But soon after we landed at Pisa airport and took the train to Florence her interest in the Italian Renaissance revived in a way that I found almost mysterious. Once I had given my lectures she threw us into a hectic round of tourist activities. Tirelessly she insisted on visiting every church and baptistry, every museum and cathedral. I was puzzled by this passion for the past until I realised that our visits to these historic sites had exposed yet another of my weaknesses.

As we took the creaking lift to the dome of Florence cathedral Elaine discovered that I was afraid of heights, a fear that I had never noticed in myself but which she immediately set out to maximise. Unsettled by the looming space below the dome, I could barely force myself from the lift. My eyes seemed unwilling to focus on the curving walls, and I felt my heart-beat fall away, leaving me on the edge of a fainting fit.

Gesticulating to Elaine, I refused to follow her around the narrow gallery. Scarcely able to breathe, I waited as she proudly circled the dome, calling to me in an insistent voice that embarrassed me in front of the other tourists. Yet as we left the cathedral she became strangely solicitous, holding my arm in a concerned and reassuring way. Far from deriding me, she seemed genuinely alarmed by my moment of panic.

Despite this show of affection, I soon noticed that our tour of Tuscany had become a series of vertical ascents. No battlement existed that we did not scale, no worn steps that we did not climb. At the Palazzo Vecchio, under the pretext of showing me the spectacular view over the city, she forced me to lean through the very windows from which Lorenzo de Medici had suspended the strangled plotters against his rule. I saw Siena cathedral from the roof down, almost breathing my last in the confined bell-tower. And all the while Elaine would watch me with her affectionate and lingering smile, like an older sister observing a timid sibling. Was she trying to cure me of my fear of heights, or to rub in my sense of my own inadequacy?

A climax of sorts came at San Gimignano, that surrealist township of towers constructed during the 14th century by rival families within this independent city state. As Elaine moved tirelessly from one tower to the next, I retreated to a café beside the cathedral with its macabre images of hell. All afternoon she gazed at the towers, admiring these symbols of an erect masculinity of which her husband was incapable, then sat beaming at me as the tourist coach carried us to Florence.

Three days later, when we arrived in Pisa for our London flight, I had been routed by Elaine's campaign. We were both eager to return to England, I to the safety of my university office, she to her solicitor. We had packed in silence, and reached Pisa airport with two hours to spare before our flight. Inevitably we found ourselves taking a taxi into the city. Reading from her guide-book, Elaine described the baptistry and cathedral in glowing terms, but I knew that our real destination was the nearby campanile, this marble phallus that seemed to excite her even more than the towers of San Gimignano.

I stepped from the taxi and stared up at the dizzying structure with its dangerously canted floors. Without a word, Elaine strode away from me towards the tower. She paid her entrance fee and began to climb the steps behind two uniformed sailors and a father with his daughter. As she reached each tier she looked down at me with her affectionate but knowing smirk, her contempt rising with each successive storey.

I stood on the cathedral steps, still surprised by the steep inclination of the tower, some 17 feet from the vertical. Despite myself, I wished that the structure, tilting each year by a few added millimetres, would decide on this exact moment for its long-predicted collapse.

Then, as Elaine reached the penultimate tier, I found myself needing to touch the tower, to feel the unforgiving marble against my skin. I left the cathedral and walked across the worn grass where the tourists sat in the sun, waving to their friends high above them. Ignoring the ticket office, I strolled around the stone well that surrounded the tower. I placed my hand on the antique marble, its surface pitted with the graffiti of centuries, its veins as marmoreal as fossilised time. The tower was both too erect and too old. I pressed against the massive flank, urging it on its way.

Eight storeys above me, Elaine had reached the roof and stood beside the panting sailors. Scarcely out of breath, she seized the iron rail and smiled down at me in her most implacable way, slowly shaking her head at my weakness.

Angered by her open contempt, I pushed again at the solid marble. The wall refused to yield, but when I lifted my hand I noticed that a small crack had appeared in the surface, running away from a discoloured node of crushed limestone. Curious, I pressed again, only to see that the crack had widened. It inched upwards at a barely visible pace, then darted forward, climbing the wall like a sudden fissure in a sheet of ice. Three feet long, it crossed a decorative moulding and rose swiftly towards the cornice of the first tier.

Laughing at this, I pressed both hands at the marble drum. Immediately the crack accelerated, and I heard a distant rumble, the dark groan of an awakening creature deep within the tower. The crack was now an open fissure through which I could see the shoes of the startled old man resting before he and his wife made their way to the second storey. A fine rain of dust and crumbling mortar showered my face. The entire tower was trembling against my hands, and a section of cornice fell through the air, followed by a scatter of fragments each larger than my fist.

The Tower of Pisa was about to fall. I gave it one last push, both arms outstretched, and felt the tortured rumbling as somewhere the spine of this great edifice began to crack. I stepped back, aware that the building was about to collapse on to me, and then looked up at the roof, where Elaine was clinging to the iron rail.

The tower buckled, its columns spilling like skittled pins at a bowling alley. In the last moments, as Elaine was pitched over the rail, I saw her face falling towards me, and an expression of anger that unmistakably changed, as she noticed me far below her, to one of triumph.

* * *

A second Tower of Pisa is now rising on the site of the first, financed by the world-wide appeal launched soon after the tragedy. The structure, this time mounted on an immovable concrete base, has reached the third storey and already reveals the modest inclination designed into it. This tower, supported by a rigid steel armature, will never fall, and within a few decades most visitors will have forgotten that it is no more than a replica.

For me, though, the original tower remains as real as ever in my mind. I often wake from terrifying dreams as the tons of marble hurtle towards me. Then I remind myself that it was Elaine who died on that day. I remember the expression on her face, the fierce pride that lit her eyes.

Did she feel that she had at last triumphed over me, and was happy to see me crushed by the cascade of tumbling columns? I remember the stones pelting my shoulders while I tried vainly to step back from the tower. At the last moment, as an amateur video-film reveals, the structure seemed to buckle, twisting itself in a desperate attempt to remain upright. It slewed away from me, sweeping Elaine, the collapsing masonry and the cartwheeling columns towards the ground by the cathedral steps.

I escaped, but that expression of triumph on Elaine's face still puzzles me. Had she seen me pushing against the tower and assumed that I was responsible for its collapse? Was she proud of me for hating her so fiercely, and for at last stirring from my impotence to take my revenge? Perhaps only in her death did we truly come together, and the Tower of Pisa served a purpose for which it had waited for so many centuries.

Copyright © JG Ballard. All rights reserved

• The standfirst on this story was amended on Thursday 30 April 2009. The previous version said that The Dying Fall was JG Ballard's last short story. Rather, it had never appeared in any Ballard collection and was not widely known. It was published in April 1996 in a UK science fiction journal, Interzone. This has been corrected.