The skeleton of a Roman woman and the bones of the mouse and shrew that gnawed her ankle in her coffin, inspiring one of Sylvia Plath's most haunting poems, have gone on display.

Plath saw the massive stone sarcophagus and its contents soon after it was excavated in the 1950s, when she was a student at Cambridge.

Staff at the university's Museum of Archaeology and Anthropology mounted the rodent bones on a piece of card – also on display again – and showed them in the coffin alongside the remains of the middle-aged woman, which is grimacing as if in pain.

The viewing prompted Plath's 1957 poem All the Dead Dears, in which she describes "this antique museum-cased lady" and the "gimcrack" bones of the rodents "that battened for a day on her ankle-bone", and fears that the "barnacle dead", strangers or members of her family will drag her down and suck her life away. Six years later, the poet killed herself.

The sarcophagus, with its inner lead coffin, was one of a group of high-status burials discovered by chance by builders clearing land for a housing estate at Arbury, on the outskirts of Cambridge.

The curator, Mark Elliott, said: "People see the wealth of finds from Arbury, and the classical imagery and Latin inscriptions all over our buildings, and think – wow, Roman Cambridge must have been really something. But the truth is, the money was at places like Arbury, from anything we've found Roman Cambridge was a crossroads if that, a Little Chef halt on the main road to somewhere more important."

The sarcophagus was taken off display a generation ago due to overcrowding. Donations to the museum, founded in 1884, have poured in from academics and wealthy former students around the world, who have sent crates containing items such as a South Seas dragon with Chinese tea bowls for eyes; a group of freeze-dried 500-year-old potatoes from Peru; and a mysterious object believed to be a Viking ironing board.

Other unusual Roman finds include a pot considered so obscene by Victorian excavators that it was not listed among the discoveries from the site in Great Chesterford, Essex. The pot, believed to have been made near Peterborough, shows naked women driving a carriage pulled by four penises, with phalluses floating overhead.

Speculation that the item was a good luck symbol or fertility charm have been put forward, but the museum's director, Nick Thomas, is convinced the pot was created as a bawdy joke. "Other almost identical pots, but with conventional versions of the scene, are known. I think you'd have put the two side by side and people would have roared," he said.

Although twice as many objects are on display in the remodelled galleries, they represent less than 1% of the collection.

The museum's most precious new acquisition is a front door, the first in its 99-year history on the present site. The old entrance was a narrow, arched doorway in a turret in the corner of a courtyard. "The old entrance had its charm," Elliott admitted, "but we don't want to be Cambridge's best-kept secret anymore."

The Museum of Archaeology and Anthropology, Cambridge, free from Tuesday to Saturday from 25 May