London’s Cannon Street is a frantic mêlée during the morning rush hour. As commuters hurry to work, few notice the small crypt, with a glass encasement within it, built into the wall of 111 Cannon Street. Fewer still know that to peer inside would reveal a stone – nothing extravagant, just a lump of oolitic limestone.

There are no precious metals or engravings; it’s not a dazzling artefact you might find in a museum. But what it is, and has been as long as records exist, is a literal and metaphorical part of London. Some 18th-Century writers even suggested that, much like the palladium that protected the city of Troy in Ancient Greek mythology, the stone’s survival is key to the continued existence of London itself.

“It’s always there, and always remains the same,” said Roy Stephenson, London’s Historic Environment Lead at the Museum of London. “It has stood roughly in the same spot, while everything around it has changed.”

You may also be interested in:

• What’s in London’s secret green huts?

• New York’s cheeky symbol of defiance

• How France created the metric system

To this day, the exact origin of this 53cm-by-43cm-by-30cm piece of rock, known as London Stone, remains a mystery. Studies undertaken in the 1960s revealed it was likely Clipsham limestone, probably extracted from the band of Jurassic-era rock that runs from Dorset in England’s south-west to Lincolnshire in the north-east. In 2016, results from tests conducted by the Museum of London Archaeology suggest London Stone could be from the Cotswolds, 160km west of London.

Science just can’t explain it – this is one case where archaeology has failed

London Stone was included on the earliest printed maps of the city in the 16th Century. In 1578, L Grenade, a visiting Frenchman, described it as ‘3ft high, 2ft wide and 1ft thick’. What remains today is only a fraction of the original stone that was once embedded in the ground in the centre of Candlewick Street, now known as Cannon Street, a short walk from the Tower of London. John Stow, a 16th-Century London historian, wrote in 1598: “It is so strongly set, that if carts do run against it through negligence, the wheels be broken, and the stone itself unshaken.” It was an entirely impractical position, no doubt, but bearing how much the topography has changed in London over the last millennium, it’s fair to assume that the streets were built around the stone. But that is all we can say definitively.

“Science just can’t explain it – this is one case where archaeology has failed,” said John Clark, curator emeritus at the Museum of London.

The stone has been a London landmark for centuries and has borne witness to some of the city’s most dramatic moments. On 2 September 1666, a fire broke out in a bakery on Pudding Lane. Over the next three to four days The Great Fire of London ravaged the medieval heart of London, destroying more than 13,000 buildings – including those surrounding London Stone. Its position in the middle of the street likely saved the stone from significant damage, but the inferno led to a startling discovery.

As architects began reconstructing the city, surveyors found that much like an iceberg, the visible stone was only a small portion of a much larger structure. The ‘root’ of the stone extended around 3m down into the earth. It could have been “a kind of Obelisque,” noted Robert Hooke, from the Royal Society, the UK’s science academy, at the time of excavation. This theory was supported by 17th-Century architect Christopher Wren who, through his son, Christopher Wren Jr, later speculated that it could have been “in the manner of the Milliarium Aureum, at Rome”, an ancient monument from which all roads in the Roman Empire began and mileage throughout the empire was measured.

Sadly, evidence to support the theories that it formed part of a structure dating from the Roman Empire are sparse and far from conclusive. What’s more, the stone would soon be relocated, and the construction of the Metropolitan Railway in the mid-19th Century led much of the stone’s original foundation to be quarried away.

By 1742, with London’s increasingly traffic-clogged streets, the stone had become a hazard and was moved a short distance from the centre of the street to the curb side, and placed by the wall of St Swithin’s Church. In 1940, the stone once again survived devastation after the church was all but destroyed by German bombs during the Blitz. In 1962, the remains of the church were replaced by the office building at 111 Cannon Street – which included a specially designed place to keep the stone – and it has remained here ever since (with the exception of the two years between 2016 and 2018, when it resided at the Museum of London while the building was renovated).

It’s always there, and always remains the same

Where facts and science have failed to provide a definite history, myths and stories have flourished. In 1450, an armed revolt rose against the deeply unpopular King Henry VI, whose fight to retain control of France was seen as the leading cause of England’s rising debt. Legend has it that the revolt’s leader, Jack Cade, placed his sword upon London Stone and declared himself ‘Lord of London’, an event that was dramatised – and much exaggerated – in Shakespeare’s Henry VI Part 2. Despite the effect this must have had on audiences, there is no evidence of the stone being employed for such declarations, before or after Cade.

While it may not have been used to overthrow the monarchy, London Stone has played a role in enforcing royal decrees. One particular incident comes from The Worshipful Company of Spectacle Makers, an organisation founded in 1629 by royal charter with the purpose to ‘support the prevention and treatment of vision impairment’ – a cause the company champions to this day. If spectacles in the city were found not to meet the requirements set out by the company, a claim was brought to court where a guilty verdict would result in punishment or destruction of the eyewear. Minutes from a 1671 case stated that 264 spectacles ‘were found badd and deceitful and by judgement of the Court condemned to be broken, defaced and spoyled both glasse and frame... on the remayning parte of London Stone’.

“We think it was a well-known landmark, a gathering place and the point where people travelling into the City would know they had reached London,” said Helen Perkins, Clerk (CEO) of The Worshipful Company of Spectacle Makers, noting that the destruction of the spectacles upon such a famous landmark acted as public deterrent.

Although the stone’s original purpose may be lost, its symbolic importance is undeniable. Through fire and bombs, from Roman legionnaires to civilian revolutions, London Stone has remained ever-present.

We may not know what it was or where it came from, but we dare not move it now – the future of London might just be at stake. But then again, maybe it’s just a stone.

Join more than three million BBC Travel fans by liking us on Facebook, or follow us on Twitter and Instagram.

If you liked this story, sign up for the weekly bbc.com features newsletter called "The Essential List". A handpicked selection of stories from BBC Future, Culture, Worklife and Travel, delivered to your inbox every Friday.