The disappearance of Rome's Ninth Legion has long baffled historians, but could a brutal ambush have been the event that forged the England-Scotland border, asks archaeologist Dr Miles Russell, of Bournemouth University.

One of the most enduring legends of Roman Britain concerns the disappearance of the Ninth Legion.

The theory that 5,000 of Rome's finest soldiers were lost in the swirling mists of Caledonia, as they marched north to put down a rebellion, forms the basis of a new film, The Eagle, but how much of it is true?

It is easy to understand the appeal of stories surrounding the loss of the Roman Ninth Legion - a disadvantaged band of British warriors inflicting a humiliating defeat upon a well-trained, heavily-armoured professional army.

It's the ultimate triumph of the underdog - an unlikely tale of victory against the odds. Recently, however, the story has seeped further into the national consciousness of both England and Scotland.

Find out more Rome's Lost Legion is on the History Channel on Thursday 17 March at 2200 GMT

The Eagle is in UK cinemas from 25 March

For the English, the massacre of the Ninth is an inspiring tale of home-grown "Davids" successfully taking on a relentless European "Goliath". For the Scots, given the debate on devolved government and national identity, not to say the cultural impact of Braveheart, the tale has gained extra currency - freedom-loving highlanders resisting monolithic, London-based imperialists.

The legend of the Ninth gained form thanks to acclaimed novelist Rosemary Sutcliff, whose masterpiece, The Eagle of the Ninth, became an instant bestseller when published in 1954.

Since then, generations of children and adults have been entranced by the story of a young Roman officer, Marcus Aquila, travelling north of Hadrian's Wall in order to uncover the truth about his father, lost with the Ninth, and the whereabouts of the Legion's battle standard, the bronze eagle.

The historians have dissented, theorising that the Ninth did not disappear in Britain at all, arguing both book and film are wrong. Their theory has been far more mundane - the legion was, in fact, a victim of strategic transfer, swapping the cold expanse of northern England, for arid wastes in the Middle East. Here, sometime before AD 160, they were wiped out in a war against the Persians.

Image caption The Ninth would have spent much of their time at York

But, contrary to this view, there is not one shred of evidence that the Ninth were ever taken out of Britain. It's just a guess which, over time, has taken on a sheen of cast iron certainty. Three stamped tiles bearing the unit number of the Ninth found at Nijmegen, in the Netherlands, have been used to support the idea of transfer from Britain.

But these all seem to date to the 80s AD, when detachments of the Ninth were indeed on the Rhine fighting Germanic tribes. They do not prove that the Ninth left Britain for good.

In fact, the last certain piece of evidence relating to the existence of the Legion from anywhere in the Roman Empire comes from York where an inscription, dating to AD 108, credits the Ninth with rebuilding the fortress in stone. Some time between then and the mid-2nd Century, when a record of all Legions was compiled, the unit had ceased to exist.

But what happened to the Ninth?

Theories on the Ninth Ambushed in Caledonia while fighting revolt

Destroyed in the Bar Kokhba Jewish revolt

Wiped out in battle against the Parthians

The early years of the 2nd Century were deeply traumatic for Britannia. The Roman writer Fronto observed that, in the reign of the emperor Hadrian (AD 117 - 138), large numbers of Roman soldiers were killed by the British.

The number and full extent of these losses remain unknown, but they were evidently significant. The anonymously authored Augustan History, compiled in the 3rd Century, provides further detail, noting that when Hadrian became emperor, "the Britons could not be kept under Roman control".

The British problem was of deep concern to Roman central government. Thanks to a tombstone recovered from Ferentinum in Italy, we know that emergency reinforcements of over 3,000 men were rushed to the island on "the British Expedition", early in Hadrian's reign. The emperor himself visited the island in AD 122, in order to "correct many faults", bringing with him a new legion, the Sixth.

The fact that they took up residence in the legionary fortress of York suggests that the "great losses" of personnel, alluded to by Fronto, had occurred within the ranks of the Ninth.

Image caption Archaeological evidence of the legion's fate is scarce

It would seem that Sutcliff was right after all.

It was the Ninth, the most exposed and northerly of all legions in Britain, that had borne the brunt of the uprising, ending their days fighting insurgents in the turmoil of early 2nd Century Britain.

The loss of such an elite military unit had an unexpected twist which reverberates to the present day. When the emperor Hadrian visited Britain at the head of a major troop surge, he realised that there was only one way to ensure stability in the island - he needed to build a wall.

Hadrian's Wall was designed to keep invaders out of Roman territory as well as ensuring that potential insurgents within the province had no hope of receiving support from their allies to the north. From this point, cultures on either side of the great divide developed at different rates and in very different ways.

The ultimate legacy of the Ninth was the creation of a permanent border, forever dividing Britain. The origins of what were to become the independent kingdoms of England and Scotland may be traced to the loss of this unluckiest of Roman legions.

Dr Miles Russell is a senior lecturer in Prehistoric and Roman Archaeology at Bournemouth University.