On the afternoon of Thursday 16 October 1834, John Snell, an ironmonger from Devon, was enjoying the sights of London. He and the friend with whom he was staying took a tour of the Houses of Parliament, guided by the deputy housekeeper Mrs Wright. Snell had never seen the House of Lords before but, as he later explained to a government inquiry, he thought it "a strange thing that there should be so much heat and smoke in it". Wright, whose complacency on that day was the cause of much later comment, assured them there was nothing to worry about.

Three hours later the Lords was unmistakably on fire. At 6.30pm a great mass of flame exploded out of the side of the building towards Westminster Abbey and another from the roof. For the rest of that brightly moonlit night a vast crowd watched the last great show of Georgian London. A sublime pyrotechnic display, it destroyed the medieval Palace of Westminster, home to Parliament for eight centuries, and marked the end of an age. Spectators, including Turner, busily sketching, packed the river so tight with boats that it was possible to walk from one bank to the other. At Windsor William IV and Queen Adelaide saw the blaze on the horizon.

Tracing the events of that night hour by hour, Caroline Shenton's book recreates the suspense and drama so well that it is easy to forget that we know what happened in the end. The desperate and ultimately successful attempt to save Westminster Hall, the largest hammerbeam roof in Europe, is fought out inch by inch, as is the chaos of the first efforts to quell the fire. Among the many strange sights that evening were parliamentary records being flung from windows, while the antiquary Thomas Phillipps ran about picking them up from the cobbles. Chance, the famous fire brigade dog went bounding through the mêlée, while on the Abbey roof another antiquary, Francis Palgrave, tried in vain to persuade the Dean to move the Domesday Book further from the advancing flames.

The vividness of Shenton's description comes from the wealth of first-hand testimony she has accumulated. As well as the government enquiry, she quotes from press reports and private letters, telling the story in many voices. We hear from the famous, including Cobbett, Carlyle and Pugin, the unjustly forgotten, notably James Braidwood, superintendent of the London Fire Engine Establishment, whose expertise and bravery were largely responsible for controlling the fire, and many like Snell who remain otherwise unknown. Frances Rickman, daughter of a clerk assistant to the Commons, whose family lived at the Palace, sat down at 3.30am to write a breathless account of events, reassuring relatives in the country that she was safe, though the fire was not out and her room was lit by the "still blazing House of Commons".

Nobody doubted that the conflagration marked a turning point. The early 1830s had seen widespread civil unrest and agitation for reform. Cobbett was not alone in seeing in the smoking ruins "a well-merited visitation" on a corrupt regime. Yet as Shenton emphasises, the Whig government then in power had brought in many of the overdue reforms. It had extended the franchise, passed the Factory Acts – indeed in closing one old nest of sinecures, the Exchequer of Receipt, it had, indirectly caused the fire. The wooden tally sticks that had been used as receipts were now redundant and it was the burning of them that had melted the copper flues under the Lords' Chamber and started the blaze.

In one sense the cause hardly mattered. The palace, adapted, extended and infilled over centuries had been, since the Reform Act, crowded to bursting point. Six hundred and fifty-eight MPs were packed like "herrings in a barrel". Space was at such a premium in the Commons that members had been known to vote with the opposition rather than give up their place. That the building was a tinder box had often been pointed out and the problem of how or whether to rebuild it had been much debated. That question at least was settled by the fire. Yet the loss to history and architecture was immense. The painted chamber, one of the wonders of medieval Europe, was destroyed, as was St Stephen's Chapel, where Parliament had sat since the Reformation, along with innumerable historic records.

Not everyone was sorry. The king seemed "not much affected" as he and Adelaide inspected the ruins, while officials placed the soggy remnant of a red carpet in front of them, snatching it up as they passed to relay it again in front. Charles Barry viewed the blaze with a professional eye: "What a chance," he remarked "for an architect." Some people thought that if the unpopular new Poor Law had been burned it was no longer the law, while those who hired out cabs or ran pubs in the area enjoyed bumper takings, as did pickpockets. The wider consequences took time to be felt, but they were tremendous. The need for a single public fire brigade was established. A national debate about architecture, gothic versus classical, erupted in the press as the possibilities for a replacement building were considered before Barry did eventually get the job. More generally, a new order was established in political life. The king's peremptory decision to dissolve Parliament that December was a last overplaying of the royal hand. No monarch ever exercised the prerogative again.

The new palace embodied a dawning Victorian age of order. The archivist Henry Cole, later the mastermind of the Great Exhibition, catalogued more than 4,000 parchment rolls of ministerial accounts and stored them in strong boxes. Record keeping, fire precautions and a further flushing out of sinecures ensued. The old palace was, to a surprising degree, wiped from historic memory. One of the many achievements of Shenton's scholarly but gripping account is to revive, in all its intricacy and richness, the ghost of one of London's greatest lost treasures.

• Rosemary Hill's God's Architect: Pugin and the Building of Romantic Britain is published by Penguin.