I must have walked down the via Nationale in Rome dozens of times. Most tourists to Rome pass this way - it’s the main street which runs down from the square in front of Termini railway station towards the Forum and the Piazza Venezia. It’s a broad but nondescript 19th-century avenue lined by rather ordinary hotels and shops.

But there is also a church, St Paul’s within the Walls - the first Protestant church to be built in Rome - which I’d never given a thought to until I visited this summer.

I was following the footsteps of the artist Edward Burne-Jones who is the subject of a major retrospective which opens tomorrow at Tate Britain. Burne-Jones - a lifelong friend of, and frequent collaborator with William Morris - visited Rome in 1871 at the end of a tour of Italy.

He came to sketch and see some of the most famous ancient sites - including the Colosseum, Pantheon, the Forum. And also to study Renaissance art. The Sistine Chapel ceiling was a particular revelation for him - he lay on the floor for hours and contemplating Michelangelo’s use of colour and his extraordinary powers of design through through a pair of opera glasses.

But Burne-Jones was also to make a major artistic mark of his own on the city. In 1881, he was commissioned by the English architect, G.E. Street to design and install a series of mosaics in the apse and choir of St Pauls within the Walls.

The Pantheon Credit: Getty

The result counts as his largest - and also one of his greatest artistic achievements. Ironically, he never saw the resulting mosaics, nor even the church - it was built five years after his 1871 visit. They were made from cartoons which the artist sent to Venice where the coloured glass tesserae were made and laid out on the drawings.

It was his assistant, Thomas Rooke, who travelled out to Rome to oversee the installation in the church. When Burne-Jones died in 1898 at the age of 64, he hadn’t managed to make it back to Rome. (One final mosaic - The Fall of the Rebel Angels - was never installed and an oil sketch is currently in the Tate Britain exhibition).

The scheme of the four completed mosaics is theologically quite complex, but the visual impact of the tesserae which fill the ceilings above the altar with shimmering light and colour is a powerful one, and there are moments of extraordinary beauty. An Annunciation which places Mary and Gabriel against a desert sunset is particularly powerful.

And I also warmed to the Tree of Forgiveness, which suspends Christ in crucification pose, but before the Tree of Knowledge rather than nailed to the cross. Adam stands to one side and Eve on the other, cast not at the originator of Sin, but as a mother caring for her two children.

Adam stands to one side and Eve on the other, cast not at the originator of Sin, but as a mother caring for her two children in this work by Edward Burne-Jones Credit: Getty

As well as this unexpected artistic discovery, following the Burne-Jones trail also led me to two other Roman sights which proved just as revelatory. One was the Domus Aurea, which had long been on my wish list, but I’d never managed to get into. The other was the Baths of Caracalla, which I’d always discounted in favour of so many other better-known ruins.

How wrong have I been? The baths are a little way outside the city centre, about 20 minutes’ walk south of the Forum. They turned out to be an astonishing site. Started by one emperor, Septimus Severus in 212AD and finished by his son, Caracalla, in 217, they were were one of the most ambitious building projects of Ancient Rome. The monumental towers at the entrance were over 100ft high and the diameter of the dome above the circular Calidarium (hot bath) was only a little smaller than the record-breaking Pantheon.

The baths of Caracalla Credit: istock

The red-brick ruins still stand in their own extensive gardens. Of course, all the roofs have collapsed, and many of the best finds have been shipped out to museums. But the walls survive to a great height, and you can still clearly see the key halls and bathing pools, and the remains of once fabulous decorations - marble facing, the sculpted cornices and friezes, and the floor mosaics.

Some of the detailed survivals are simply extraordinary, such as a gaming board carved into a marble slab by the entrance to the Olympic-sized swimming pool (natatio). It was the details which also fascinated Burne-Jones - his notebooks show sketches of the design of the gymnasium mosaics for example, and the architectural details of alcoves and stairways. Best of all from a tourist’s point of view though was that on a glorious August morning, while thousands sweated in the Colosseum queues, there were only perhaps 30 or 40 other visitors enjoying it with me.

The Domus Aurea (also known as the Golden House, or Nero’s House) is an altogether different experience and equally neglected by tourists. It was discovered accidentally in 1495, at the height of the Renaissance, when an unsuspecting peasant fell down a shaft in a field opposite the Palatine Hill. He landed on a pile of rubble and found himself in a room full of fabulous frescoes. Such was the excitement that a succession of artists were lowered down on ropes to visit for themselves - including Raphael and Pinturicchio, and probably Michelangelo.

The “house” turned out to be the remains of the huge pleasure palace built by Emperor Nero between 64-68AD and embellished with a famous 120ft bronze-gilt statue of himself. But Nero was so hated for his cruelty that his successor, Trajan, tore down the statue, demolished the upper rooms and filled the lower storey with rubble as foundations for a bath complex (which has now been lost).

When Burne-Jones visited, it must have been a real, torchlit adventure - his sketches show mysterious doorways and dark corridors. But that lower storey has been steadily excavated and proper lighting installed so that you can now walk deep into the complex through dozens of frescoed rooms, terraces and courtyards.

The Domus Aurea is neglected by tourists in Rome Credit: Getty

True, the colours which decorate most of the walls have lost some of their lustre, but no other site, not even Pompeii or the Villa Poppea, gives you quite such an impressive sense of actually being inside a Rome building, one which is now more than 2,000 years old. Strange how following the art of one painter, can lead to the discovery of so much more.

Essentials

Edward Burne-Jones is at Tate Britain (tate.org.uk) until February 24, 2019§. Admission £18.

St Pauls within the Walls (the American Church in Rome) is generally open to visitors and worshippers during the day (stpaulsrome.it).

Caracalla Baths - open daily from 9am, closing time varies according to season (check on 00 39 06 399 67 700), admission €8.

Domus Aurea - open weekends 9am-4.15pm, admission €14 with guide and a virtual reality tour (book at coopculture.it).