When Sir Basil Spence designed Coventry Cathedral in 1956, his architectural vision was about a resurrection of faith – and the city's fabric, which had experienced something like the fires of hell on the night of 14 November 1940, when a Luftwaffe incendiary raid left St Michael's Cathedral and swathes of the city centre in ruins. Nearly 70 years on, resurrection is still on the agenda here. This time it's called urban regeneration, in a metropolis where postwar experiments in city planning produced Britain's first shopping precinct, and the infamously hazardous switchback ride otherwise known as the inner ring road.

The opening of the new £11.5m extension to the Herbert Art Gallery and Museum, designed by architects Pringle Richards Sharratt, is the latest chapter in the transformation of key sites in the city centre. The Belgrade Theatre extension by Stanton Williams is a quietly graphic abstraction of surfaces and colours; the Lower Precinct shopping centre by Michael Aukett Architects is relatively clean limbed; Arup Associates is designing an engineering and computing faculty at Coventry University; and MacCormac Jamieson Prichard – which designed the BBC's new building in Portland Place, London – has transformed the three-hectare site that connects Coventry Cathedral with the Transport Museum, injecting the city with its most significant public art since the vivid creations of John Piper, Graham Sutherland, Jacob Epstein and John Hutton (not to mention engineer Ove Arup's beautifully tapered ferroconcrete pillars) gave the new cathedral an extraordinary grace.

The revived Herbert gallery and museum treads an interesting line between the past and the future on a site that has taken its architects and the planning department of Coventry City Council more than five years to unlock. The new building is shoe-horned between part of Coventry University's city-centre campus, and the segment of originally medieval streets in which lie the remains of St Michael's, and Spence's cathedral.

Download the new Independent Premium app Sharing the full story, not just the headlines

The gridshell roof of the Herbert – a cap doffed, so to speak, to the cathedral's marvellous roof structure – rises and falls like a languidly breaking wave over the museum's new arcade and history centre, its fine, gutterless edge stopping a few metres short of a line of rust-patinated Cor-Ten steel panels that jut up along the edge of Bayley Lane. Public art? Not at all. Public history, and a key part of the vision of John Pringle, an architect whose approach to the design of the new Herbert has conflated hi-tech details with impressions of place, and time. Mind you, there's been consternation about the panels: "Don't forget," said Pringle, "this is a town where rust signifies that something's gone badly wrong!"

But those steel panels mark, very precisely, the procession of medieval building plots that once stood in Bayley Lane; William Coton, a mercer, was one of its inhabitants; a century before him, on the same site, lived one Robert Allesley, a girdler. Fifteen metres and five centuries away, in the Herbert's new arcade, stands a Victorian Jaquard loom whose ectoplasmic splays of thread and bolus hooks prefigure Coventry's 20th-century rise as Britain's greatest car manufacturer, machine-tool maker and producer of new-fangled white goods. The passage of time is palpable here, and even more potent in the preserved remains of the gothic Hammer House of Horror vaults of one of the medieval buildings preserved under the Herbert's forecourt.

If you follow the edge of the row of panels, they take the eye, on one side, to the cathedral; and, on the Herbert side, to The Enfolding, a charmingly modest statue by Jean Parker dedicated to peace, reconciliation and the announcement, for some reason, of the artist's BA Hons status. But never mind that: 60 metres beyond her artwork lies pure, buttock-clenching architectural schizophrenia: the ludicrously overscaled proscenium arch bolted on to the facade of Coventry University's gateway building succeeds only in accentuating the very thing it's supposed to hide – an out-and-out minger of a building from the 1960s.

Pringle's architecture is also, in part, a bolt-on whose design risks two things: structural overstatement and a blurring of what architects would refer to as its typology. If you encounter a big, columned classical facade in a large town or city, the typology in question will be Important Civic Building; if you wander through parts of Shoreditch and Hoxton in London, the typology of the big tough four-square brick buildings will be 19th-century industrial. Viewed from the entrance of Spence's cathedral, the typology of the new addition to the Herbert seems uncertain. The form doesn't send an obvious message about its art or historical function.

Yet it is this uncertainty which highlights the key thing about this building – the way it creates a processional space. The big arcade linking the new galleries and the museum is full of light; it feels utterly open-handed, a public extension not just of the Herbert but of the university's central plaza and the precincts of the cathedral. It's a place rather than simply a building, and one where the architect and the landscape designer, Edward Hutchison, have managed an even more significant kind of blurring, a segue between indoor and outdoor, and between one kind of activity and another. There's a relaxed vibe about it.

"When we first examined the site, looking from the cathedral, the space leaked away into a car park," said Pringle. "Projects like this must add something to the public realm. Our proposal to demolish an existing wing of the Herbert, to create something that faced Bayley Lane and the cathedral, was controversial. But the planners, and the city planner John McGuigan in particular, were open-minded enough to take on board something that could add something that was distinctive and valuable to this part of the city."

The arcade connects the core of the original museum and gallery spaces with the new galleries; and here, Pringle has created calmly luminous volumes by using white, quartz-mixed concrete. His treatment of details signals his early career with Hopkins Architects, whose fusion of hi-tech structures and brilliantly crafted connections – particularly between timber and steel – have become an almost unchallenged trademark.

We see the Hopkins imprint on the details of the diagrid roof, and on the super-svelte staircase that rises up the side of the gallery's arcade wall. But in an unexpected move, Pringle has twisted the structural grid just at the point where the arcade's roof swoops down to become the roof of the history and archive centre, so that its timber beams pick up the alignment of those Cor-Ten medieval markers. This admirable transition is one for architectural anoraks. Everything's at stake: changes of scale, materiality, thrust, threshold, detail. It could have been an architectural train-crash. It isn't.

But these are design niceties. The architecture's most significant contribution is in the way it has added something humane and civil to yet another moment of urban and cultural resurrection in Coventry. The arcade will no doubt come to feature on Coventry postcards; it may even win Pringle Richards Sharratt an award or two, and interest the ghost of Ove Arup. What really matters, though, is that the new architecture of the Herbert works as a piece of urban connective tissue. Not the sexiest of phrases, to be sure, but one that Coventry, coolly and without fuss, is taking very seriously.