Ridley Scott claims his new film starring Russell Crowe will be the most historically accurate ever. But what do we actually know about the real outlaw and his merrie men?

'Robin Hood was almost certainly a pedestrian," David Crook, the retired former assistant keeper of public records at the Public Record Office, tells me over tea one afternoon at his home in Grantham. Robin, in other words, had no horse. This is significant, because, as I settle down to try to unravel the eight centuries of myth and legend that have accreted around the outlaw, I am looking at a still from the new Ridley Scott movie, which will open the Cannes film festival on 12 May. Russell Crowe – looking the spit of Maximus, the hero of Gladiator, with cropped hair, bloodied cheek and an expression of furious determination – is astride a horse. The horse, naturally, is white: what else would a hero, about to save England from French invaders, ride? I fear there may be some historical disconnect here.

Scott has told Empire magazine that history is at the heart of his thinking. "It's always stronger if you do that . . . I always like to know where they're coming from." Scott's Robin has been fighting alongside King Richard (the Lionheart) in the Crusades – but the entire Crusades backstory that now dominates treatments of Robin was, in fact, established in the public imagination by Ridley's namesake, Sir Walter Scott, in his novel Ivanhoe (1820). It has given an essentially local tale – Robin's banditry somewhere in northern England – a context never intended in the 15th-century ballads on which the Robin Hood mythos is based.

The echo of Maximus is heard throughout the new film: wronged son returning to claim what is rightfully his and, in so doing, uphold the honour of the dead Good King Richard, oppose the depredations of Bad King John, beat off the French, and prepare the land for Magna Carta and a blissful future, winning the hand of Cate Blanchett's feisty Maid Marian into the bargain. It condenses 20 years of history (c 1199-1218) into a couple of hours, and puts Robin centre stage in a story in which he had no part at all.

I would be less harsh on the new movie were it not for the exaggerated claims made on its behalf by director and star. Scott has said the only Robin Hood film he really liked was Mel Brooks's spoof, Men in Tights. The rest, he dismissed, saying: "We're at another level." Crowe has made similar claims: "When the first script came to me, I said I'd do Robin Hood, but I wanted to do a fresh version where we revitalise every part of the story," he told the Sunday Times. "If you're going to revitalise Robin Hood, it can be done on the basis that whatever you thought you knew about Robin Hood, it was a previously understandable mistake."

Brave words, but not matched by deeds. Ironically, the first scripts for the film, back in 2007, did provide the sort of radically different take on the Robin Hood story Crowe was looking for: Robin Hood as dodgy outlaw and a more sympathetic sheriff of Nottingham (a reversal brilliantly managed in the 1976 film Robin and Marian, starring Sean Connery as an ageing, trouble-making Robin). The 2007 treatment, by Ethan Reiff and Cyrus Voris, sounds very promising, its boldness underlined by the fact that the working title was not Robin Hood but "Nottingham". It could have been glorious, but Scott hated it.

"It was ridiculous. It was a terrible, page-one rewrite. If you're going to invest in a Robin Hood story, why call it Nottingham? You'd end up spending 80% of the publicity budget explaining why it's Nottingham, not just Robin Hood. It doesn't make any sense."

Brian Helgeland, who co-wrote the Matt Damon film Green Zone, was brought in, there were several rewrites, numerous delays, and what evolved was a much more conventional tale. For all their "Robin as you've never seen him before" posturing, Scott and Crowe have bowed down before the potency of the heroic, romantic Robin, the creation of Sir Walter Scott and the syrupy, imperialist Victorians, for whom Robin was a signifier of nationhood.

Crowe is thicker set than Douglas Fairbanks, whose 1922 Robin set the cinematic archetype, and has less brio than Errol Flynn, whose 1938 portrayal is the most glittering on screen. He plays Robin with a grimace (a dangerous departure, as the joyous freedom of life in the greenwood is part of the tale's enduring appeal), but this is still Hollywood Hood, out of Ivanhoe.

Forget it. We are going into the forest in search of something more satisfying. We are going to unearth the real Robin.

Leon Unczur, the current sheriff of Nottingham, has a wonderfully medieval face. I meet him in a cold little room full of flags, called the south bastion, next to the gatehouse of Nottingham castle. He is wearing a black gown, a gold chain and is seated on a wooden throne. They take Robin Hood seriously hereabouts. Unczur is a councillor and used to be responsible for culture in Nottingham. He is sheriff for a year, and loving every moment: Nottingham had one of those urban beaches in the city centre last summer, and he made a point of going round kicking children's sandcastles over. "I always played the baddie in school plays, so maybe I was born for it," he says.

Nottingham sees huge tourist potential in Robin, and Unczur has set up a commission to look at the possibility of building a theme park in the grounds of the castle. At the moment, there is remarkably little in the city to tie it to its global icon. There's a statue of Robin outside the castle, small and squat, not at all like the god-like Errol Flynn figure; there are posters on the big wheel in the market square proclaiming "Robin Hood month" in May; the Nottingham Building Society has Robin Hood's head, in green with that characteristic pointy hat, as its logo. But otherwise surprisingly little, given that, throughout the world, Robin and Nottingham are synonymous. "We need to do more about drawing people in because of Robin Hood," says the sheriff. "We need to make more of him."

Did he really exist?, I ask Unczur. Plenty of people think Robin is a myth, beginning life as a woodland sprite somewhere in a lost pagan past. "Somebody existed," he says. "Some, several – it doesn't really matter. They existed and they exist now. It's like any yarn: it gets spun until you turn into Errol Flynn. Who was he? Was he anybody specific? It really doesn't matter because it's the stories that people relate to."

"And the values," chips in Stephen Barker, Nottingham city council's head of communications who is with us in the south bastion, "the environmentalism, the romance, the cunning. He's victorious by being smart."

The man who used to do Barker's job, Bob White, is now chairman of the World Wide Robin Hood Society, and part of the sheriff's commission looking at the feasibility of building a Robin Hood theme park in Nottingham. He too is bothered by Nottingham's half-heartedness; less so by the question of whether Robin really existed. "The great thing about the Robin Hood legend is that, because it's so iconic, it has outgrown and overtaken all the issues which are unproven. People just like the principle of the story; what it stands for. It's like they say in the films of the 1940s: 'If the truth gets in the way, print the legend.'"

This is not what I want to hear. I am looking for the prototype of Robin Hood, the origin of the stories that have spun out of control over the centuries. Over coffee in the castle's cafe, I ask White: what is the reality of Robin? "The tale as it's come down to us is a mixture of different stories," he says. "The most common thought is that it started with the troubadours, probably as part of the May Day 'green man' idea [a spirit of the forest, clad in leaves, celebrating the coming of spring] – Rob of the Hood, or Robin the Hood, who was a significant character in medieval plays. They were taken around the country, the troubadours would add stories when they came back the following year, and so the story grew."

Somewhere along the line, Robin came to be a righter of wrongs – which he isn't in the original 15th-century ballads – and White says that has added to the potency of the myth. "One of the pieces of research we did when we first formed the society was to get in touch with a cross section of people who were trading under the Robin Hood name. I approached them and said, 'Why do you call yourself the Robin Hood Laundromat in Florida?', and people came back and said, 'Because it has a credible ring. Our customers know we are a value-for-money operation.' Charities use the name for the same reason." When Richard Curtis and Bill Nighy spearheaded the recent campaign for a "Robin Hood tax" (a levy on international currency transactions), they were summoning up a long tradition.

It's odd how uninterested the cognoscenti are in the real Robin Hood. White sees the malleability of the myth as an advantage. Rob Lutton, a medieval historian who teaches Robin Hood as part of an MA course at Nottingham University, likes the way each reworking of Robin Hood examines contemporary concerns, from the end of the Middle Ages on. "The texts that we have were composed, at the earliest, at the very beginning of the 15th century. They set the stories back in an earlier period, but they're being written for people in that century. It's a way of exploring the nature of English society in the 14th and 15th centuries."

Could it really all be based on nothing? I ask Lutton, with a hint of desperation. "There are literary-historical references in the late Middle Ages to a possible originating historical figure, and there are some things in central government records, references to an outlaw called Robert Hod. But by the 1260s these names, Hod and Robhod, are being used as popular monikers for outlaws."

So the question is, was there an archetype for all these Hods, who started to appear right across England – in Yorkshire, Sussex and Cumbria, as well as Nottinghamshire? Lutton isn't sure. "Was there a model for this cultural diffusion, or have people heard the stories and looked for an originating figure because we all want to believe that this character may have existed?"

'Welcome to Robin Hood county," say the signs as you drive along the A60 towards Sherwood Forest. Sherwood used to cover more than 100,000 acres, from the city of Nottingham up to South Yorkshire. Now it's down to about 450 acres, a landscape of ancient oaks presided over by senior ranger Paul Cook. He gives me a tour of this remnant of forest, to which visitors flock as the last vestige of the hideaway of the outlaw.The centrepiece is the Major Oak, a huge tree – propped up by girders – which stands virtually alone in a clearing. The tree is reputed to have been a meeting point for Robin and his outlaws, although as Cook points out, this 1,000-year-old oak would have been a sapling in 1200.

Visitors are sometimes disappointed by the forest: not just its restricted size, but the type of trees it contains. "If you look at Robin Hood: Prince of Thieves, they're all pine trees," says Cook. "Look at the Jonas Armstrong Robin Hood series; that was filmed in a beech forest in Hungary. We have virtually no beeches in Sherwood Forest. But people come here thinking that's what it's like. You get coachloads of Americans coming, and they'll say, 'It doesn't look anything like I imagined.'"

Cook points out that the Great North Road, connecting London with York, used to run through the forest, and offered rich pickings for robbers. Then he raises another question about the "real" Robin: was he from Nottinghamshire or Yorkshire? The former has always claimed him, but the South Yorkshire lobby is getting louder, as the naming of the airport that serves Doncaster and Sheffield after the outlaw suggests. "As a fellow Yorkshireman I support Yorkshire," says Cook, "but that's just between us." If Ridley Scott had really wanted a new take on the story, he should have called his film Doncaster.

We are meeting at the Sherwood Forest Visitor Centre, which is bracing itself for a tourist influx in the wake of the Crowe film. When Robin Hood: Prince of Thieves was released in 1991 the centre, which dates back to the mid-1970s, was overwhelmed, with almost half a million visitors over the course of the year. Nottingham is planning a new £5m centre, but the old one – with its unpretentious cafe and little shop selling bows, arrows, swords and crusader gear – will have to cope for now. The forest is home to a species of beetle, the Hazel Pot, found nowhere else in the country, and Cook envisages them being trampled to death by hordes of tourists come the summer.

In the cafe, Andy Gaunt, an archaeologist with Nottingham county council, tells me not to think of medieval Sherwood as just a forest. It was a working community, governed by forest law (a draconian code introduced by William the Conqueror to protect his hunting land). The inhabitants of areas governed by forest law loathed it, and would have been likely to lionise anyone who flouted it. It is easy to see how outlaws became heroes, and common criminals could be confused with freedom fighters.

"People were being outlawed all the time," Gaunt says. "It was quite common in medieval times – it was a violent society. The forest was seen as a wild place and after dark it would be pitch black. I wouldn't want to go there on my own (though there are parts of Nottingham I wouldn't want to go on my own now). Robin Hood in the ballads is quite a brutal character, but people didn't seem to worry about that, just as people in the Wild West didn't worry that Billy the Kid was actually a serial murderer."

So did the outlaw exist? (A question I am by now getting used to asking.) "The original ballads say there was a yeoman called Robin [usually Robyn Hode in the ballads]. Some of the stories are set in Barnsdale [in Yorkshire]; some of them in Sherwood. It could be a merging of two different legends. But forget about King Richard and King John. The only king mentioned in the ballads is an Edward, possibly Edward II, but that's early 14th century. The earliest mention of Robin Hood is in Piers Plowman in 1377, and the reference shows the rhymes were already well established then, but the earliest surviving ballads are mid-15th century."

Before I leave Sherwood, I pay a brief visit to Edwinstowe, "Robin Hood's village", according to the signs on the road as you enter. It became a tourist attraction in the 19th century as the romantic, Scott-inspired cult of Robin grew. It was claimed that Robin had married Maid Marian in the local church, and a statue of the two of them on their wedding day now graces the high street. Less elegant are Maid Marian's Hair and Beauty salon, close to the church, a souvenir shop called Robin's Den, and a fish and chip shop called Robin Hood Plaice. Codpieces a speciality.

Stephen Knight, professor of English literature at Cardiff University and one of the world's leading authorities on the literary evolution of Robin Hood, calls the search for the real outlaw "vulgar empiricism". "When the Doncaster News rings up and says 'Did Robin Hood really exist?', I have a variety of answers," he tells me. "One is, 'How do I know you exist?' or 'What do you mean exist?' But I like to say, 'Of course he exists, we're talking about him.' My view is that the empirical, real Robin Hood – like the 'real' King Arthur – is a 20th-century take on reality. Who cares if there was a real Robin Hood? There's a real myth which is living and breathing."

In his 2003 book, Robin Hood: A Mythic Biography, Knight has traced the progress of Robin – from the "bold Robin" of the medieval ballads, through the courtly, noble Robin of Tudor England, to the romantic hero of the late-18th and 19th century, and on to today's avenging action hero, envisioned first in Douglas Fairbanks's hugely successful 1922 film. For Knight, each of those creations has cultural validity. "To my mind, the real history is the constant change. Over every generation there's a new Robin Hood, and no doubt Russell Crowe is going to be one of those. Perhaps he will be the post-9/11 Robin Hood."

Literary critics and historians – Knight's "vulgar empiricists" – come to Robin from wildly different angles. "Most historians think my work is shit, real shit," says Knight with disarming honesty. David Crook, the former assistant keeper at the PRO, does not go quite that far, but he does tell me Knight has "problems with chronology". Crook is writing a book about the real Robin Hood, and says he believes he has traced him to Yorkshire in 1225. When he tells me this, it is akin to finding the holy grail.

"I'm a traitor to my own place," Crook tells me. "I come from Mansfield, but I gave a lecture there a few weeks ago saying Robin Hood was a Yorkshireman. If you believe in an original, which I do, then he was a Yorkshireman. I can make the strongest case anybody can, given how limited the evidence is, for a person called Robert Hod ('an outlaw and evildoer of our land') in 1225, who I think may be the same person as a bloke called Robert of Wetherby, who was chased by a posse of sheriff's men – specially hired men, which was very unusual – in 1225 and captured. There is a payment for a chain, to hang his body in chains, in the Yorkshire accounts for that year. We even know how much was spent on the expedition to catch him."

Crook's theory is that the tale of the pursuit of Robert Hod/Robert of Wetherby rapidly spread, carried up and down the Great North Road, and within a generation Robe Hod and Robhod had become jokey generic names for outlaw. Crook says he has even found a legal document from 1261, in which a clerk has scribbled out the offender's real name and entered the joke name.

I find Crook's hypothesis seductive. Robert Hod/Robert of Wetherby is a real figure, active in the 1220s, captured and killed by the sheriff of Nottingham (briefly holding the post of sheriff of York) in 1225, spawning a Billy the Kid-type legend that spreads all over England, becoming the generic outlaw, and producing ballads and songs which are common all over England 150 years later. The chronology of cultural diffusion feels feasible: a sliver of reality – a common outlaw in the badlands of south Yorkshire, robbing travellers on the Great North Road, with nothing to suggest his motive was anything other than personal gain and whose criminal career is rapidly, and bloodily, brought to an end – gradually becomes this all-pervading myth which eventually reaches Hollywood and the world, and endangers the very existence of the Hazel Pot beetle.

I call Wetherby town hall with the happy news that they should start building their theme park. But the town clerk, Barbara Ball, is not especially taken with the idea. "I don't think that would go down very well up here," she says. "We're not too keen on change in Wetherby." The tourist office is better disposed to the idea of claiming Robin as their own. "It would be a feather in our cap, another string to our bow," says one of the staff, showing an instant fluency in the language of Robin Hood.

My quest seems complete, and I decide to visit Wetherby to convince the locals that a Robert of Wetherby (aka Robin Hood) theme park will be a goldmine. I plan a journey that will take in some of the places mentioned in A Gest of Robyn Hode, the longest of the original ballads – in particular Wentbridge, a notorious spot for bandits in medieval times because travellers had to leave their carriages and go up the hill on foot, and the place where Crook thinks Hod did much of his robbing.

I travel to Wentbridge with David Greenwood, an amateur historian besotted with Robin Hood since his mother took him to see the Errol Flynn film more than 60 years ago. An engineer by trade, he has spent much of his spare time decoding the Gest, examining – even excavating – sites connected with Robin Hood, and publishing books with his conclusions, the central one being that the Gest was written by a 14th-century mystic called Richard Rolle.

"I hope you're not going to lampoon me," he says, when he meet in the well-heeled village of Hampole, where Rolle is buried on the site of an old priory. Certainly not: first, because Greenwood has devoted his life to the endeavour of tracing Robin Hood and discovering the identity of the writer of the Gest, but also because he has Parkinson's disease, yet has kindly come up from Nottingham with his wife to explain his theories and take me to sites mentioned in the Gest. Moreover, in his favour is the fact that the owners of Robin Hood Doncaster/Sheffield airport based their choice of name on his theories, which are expounded on a mock-scroll next to an idealised statue of Robin at the airport.

On 14 May, the day Ridley Scott's film is released in the UK, Greenwood will publish the summation of his findings in a new book, Robin Hood in Barnsdale Stood. In essence, they are that Rolle wrote the poem, and that it features not one Robin Hood but three: Robin Hood the archer (of which more anon), Robin Hood the master (based on the wealthy Nottingham merchant Sir Geoffrey Luttrell), and Robin Hood the poet (Rolle himself). Greenwood accepts that, by the 1320s, Robhod was already well established as a generic name for outlawry, and thinks Rolle applied that to himself and his companions in an outlaw band which harried Edward II in both Yorkshire and Nottinghamshire in 1322-23 before being pardoned.

This is useful for me, because I can fuse the two theories: Crook's Wetherby criminal establishes the pseudonym; Rolle adds much of the detail, the raw material from which others can flesh out the legend. It is speculative, but very clever – the three Robins cover all the bases: archery, amelioration (Luttrell left a will bequeathing much of his money to worthy causes) and art.

It does, though, leave me with a dilemma. Should I head for Wetherby or Wakefield, supposed home of the yeoman on whom Robin Hood the archer was based? Greenwood tells me he was a smallholder in the borough of Newton, just north of Wakefield, and points out that the area around Wakefield has the greatest concentration of names featuring Robin Hood in the country: Robin Hood Bridge, Robin Hood Hill, and the village of Robin Hood itself, close to Newton Hill (the modern incarnation of Newton). Forget Nottingham, forget Wetherby, this could be it.

I head for Wakefield, and a taxi driver takes me on to Robin Hood, less a village than an anonymous ribbon of houses equidistant between Wakefield and Leeds. "I don't think it's got anything to do with the real Robin Hood," he warns me, but by the end of the journey I have almost convinced him that this acme of anonymity has a deep historical significance. "People round here just aren't aware of their history," he admits.

Apart from gangs of marauding youths, some of whom are lighting fires in an alleyway – the tradition of outlawry persists! – there isn't much happening in Robin Hood when I get there. The woman in the post office also insists it's got nothing to do with Robin Hood; ditto the staff of the One-Stop convenience store and the man outside the Gardeners' Arms looks at me as if I may be mad. I have come all this way to be met by indifference.

I sit outside the Old Halfway House, a pub on the main road that marks the midpoint between Wakefield and Leeds. It appears to be closed – like so many pubs these days – and there are "To let" signs outside. The spirit of the Middle Ages seems very distant. Perhaps, after all, Robin Hood the archer never did walk this way.

Then a man emerges from the murky pub with a cigarette. It is open after all. I go inside for a shandy, and ask the woman behind the bar what the town's link is with the legendary outlaw. "Haven't a clue, I'm not from round here. Ask them, they're local," she says, pointing to four middle-aged men at the other end of the room, cradling pints and watching the racing on TV. She shouts across and puts the question for me.

"The original Robin Hood was from round here," says a burly man, the spit of Little John, who for a moment diverts his attention from the 4.20 at Aintree. "Nottingham just jumped on the bandwagon."

It may not be definitive proof, but I feel a surge of satisfaction at this small victory for vulgar empiricism, and head outside to catch the bus to Leeds.