Whenever a gap appears in the traffic roaring past the new Bloomberg building in the City of London, it is just possible to make out a little hump in the road where Cannon Street and Walbrook meet. As busy a junction now as it was 2,000 years ago, it is also one of the spots where the long buried past almost breaks through the skin of the modern city.

The history of London’s secret rivers is being celebrated in a new exhibition at the Museum of London Docklands. Its curators, Kate Sumnall and Tom Ardill, have trawled the museum stores for objects – many going on display for the first time – from a dog’s skull so polished it resembles an ivory carving to a Victorian painting of deceptively pretty houses overhanging a dock, which earned the hotly contested title of “worst slum in London”.

Ignored, abused, diverted to turn mills or feed boating and skating ponds for the leisured classes, choked with rubbish, entombed in pipes and culverts, some of the rivers survive only as a soggy stain in the clay, but others still flow unseen beneath Londoners’ feet.

Kate Sumnall with a selection of Roman writing instruments that feature in the exhibition. Photograph: John Chase/Museum of London

In Ray Street, Farringdon, there is said to be a point where you can hear the waters of the enclosed Fleet, which gave its name to the former heart of journalism, Fleet Street. In 1862, 100 yards away the river “exploded” – in the words of horrified eye witnesses – bursting through and flooding the half-built Metropolitan line.

The rivers have some devoted admirers and campaigners for their restoration, such as the Tyburn Angling Society which meets every year to hear a report that fish populations have remained stable – at zero.

The Tyburn still runs under Buckingham Palace, once so close to its floors that sewage frequently flooded the Georgian kitchens. The Westbourne surfaces to feed the Serpentine in Hyde Park. North of the kink where the streets at Oval curve to avoid the Effra, what remains of that river falls into the Thames under the MI6 spooks at Vauxhall.

The sites of the confluence of streams and rivers were always powerful places with sacred significance

Sumnall and Ardill have tramped miles through London, tracing the course of the hidden rivers, which can be seen in the humps and bumps of alleys following medieval lanes, street names (all the brooks, bourns, ditches and bridges, from Knightsbridge in the west to Shoreditch in the east), old wall signs for parish boundaries, pub and church names, and even the placement of a river god’s head high in an arch of the Bank of England.

The Guardian joined them for a walk along the route of one of the most famous rivers, the Walbrook, tamed by the Romans to run under their city wall at a spot just by a church called All Hallows-on-the-Wall. The modern Drapers and Tanners Hall mark the industries it powered, and wherever developers have punched through, the finds poured out: skulls and sandals, pins and coins, jewellery and horse shoes, pots and pans, weapons and tools.

A bone miniature fish was excavated from the banks of the Fleet. It is about 400 or 500 years old. Photograph: Museum of London

Some of these objects were undoubtedly accidental losses, bitterly regretted as they fell into the grimy stream, but many were deliberately deposited – some even ritually “killed” first, such as pins deliberately bent out of shape.

But the most spectacular find of all from its banks was no carelessly discarded bauble. In 1954 thousands queued for hours in a city still pocked with scars from the Blitz, to see the unearthed Roman temple of Mithras, now spectacularly reconstructed within the huge Bloomberg building.

At the corner of the building the bones of Londinium lie almost six metres below, but the hectic road rises slightly over the line of a Roman bridge crossing the Walbrook.

The London Mithraeum being excavated in 1954. Photograph: Robert Hitchman/MOLA

“I love this spot,” Sumnall said. “Every day thousands of Romans must have walked across that bridge, coming into the city or crossing out into the fields beyond.”

Ardill added: “These crossing places, and the sites of the confluence of streams and rivers, were always special places, powerful places with sacred significance since prehistoric times.”

Although the bank of the Thames is now well south of its Roman line, the Walbrook still dribbles onto the foreshore from a pipe just upstream of Cannon Street station, past a yard that was a dock until the 1950s and where London’s rubbish is still craned out onto barges for disposal in landfill.

Mudlarks, the licensed scavengers on the tidal shore, still find medieval and earlier pieces every week. As we stood on the slimy waterstairs – a favoured spot for a quiet drink, evidenced by the litter of bottles and cans – Sumnall produced from her bag a Roman dress pin, and the base of an expensive imported Samian ware jar, which she picked up on the shore a few days earlier. So much material is being collected from a spot the mudlarks nickname “the Roman hole” that she suspects it is being washed out of a well that became a rubbish dump more than 1,500 years ago.

The “poverty and pleasure” section of the exhibition looks at the rowdy delights of Ranelagh Gardens in Chelsea, where the boating lake was fed by the Westbourne, which flows under Knightsbridge – where the crossing was once infamous for highwaymen – Sloane Square and Chelsea Barracks.

‘A present for a good girl.’ Photograph: Museum of London

A once expensive toy, a model boat found in the barracks redevelopment, contrasts with a cheap mug with “a present for a good girl” printed on it, from the south bank far downstream at Jacob’s Island. “No little girl from Jacob’s Island ever owned a toy like that,” Sumnall said.

Jacob’s Island was where the Neckinger met the Thames in a festering slum. Typhoid and cholera ravaged the mouldering houses. Dickens portrayed it as a warren of thieves and made it the site of the murder of Nancy by Bill Sikes. Soon after the publication of the novel, the rotting slums were demolished.

“Some people have questioned whether Jacob’s Island could possibly have been as bad as it has been portrayed,” Sumnall said. “If anything the archaeology suggests it was worse.”

Both curators’ favourite river is the Fleet: so many writers lived on its reeking banks that it is one of the best chronicled of all the London rivers. In 1710 Jonathan Swift wrote in A Description of a City Shower: “Sweepings from butchers’ stalls, dung, guts, and blood / Drown’d puppies, stinking sprats, all drench’d in mud / Dead cats, and turnip-tops, come tumbling down the flood.”

The dog’s skull came from the Fleet, as did a medieval toilet seat, which will go on display for the first time with a replica so visitors get a sense of the three-seater lavatory experience.

“Some of the stories are a bit sinister,” Sumnall said with considerable relish. “There was one pub with a trapdoor at the back for loading and unloading. There’s a tale of a sailor robbed, stripped, and dumped naked into the river. Isn’t that fabulous?”

Secret Rivers is on at Museum of London Docklands until 27 October 2019. Free entry

Follow Guardian Cities on Twitter, Facebook and Instagram to join the discussion, catch up on our best stories or sign up for our weekly newsletter