A few months ago the disc jockey and radio broadcaster David Rodigan was awarded an MBE to mark his 40 years as an ambassador and proselytiser for reggae. Ram Jam, as he is affectionately known, tells this story of his trip to the palace. After the investiture he was approached by Prince Charles.

"You really love this music, don't you?" said the Prince of Wales.

"I certainly do, sir," he replied.

"So do I," said Charles. "I love Jamaica."

It is incongruous to think that the heir to the throne may own a complete set of Black Uhuru albums, but his revelation should not come as a surprise. From the shack to the stately home, Jamaica inspires extraordinary affection.

In 2004 I went to the opening ceremony of the Athens Olympics. As each team marched into the stadium, a smaller or larger cheer went up, in a kind of popularity contest for the world's nation states. When the gold, black and green of Britain's former colony fluttered into view, the reception was euphoric. Of course the noise was partly for its athletes, whose explosive performances have illuminated the Games since London 1948. And yet there was something more to it than that. What was it that made this huge international crowd rise up in salute?

Jamaica is everywhere this year. In April, the musical legacy of its most famous son, Bob Marley, was given fresh allure by British director Kevin Macdonald's outstanding documentary Marley. There are two further film dramatisations of Marley's life in production. Tomorrow the country will begin a series of celebrations at home and around the world to mark 50 years since its independence from Britain. Macdonald is also executive producer of One People, a specially commissioned film in the style of his crowd-sourced Life in a Day (2011), that fuses hundreds of video snapshots of Jamaican life around the world. Usain Bolt and the powerful team whose predecessors provoked that Athens roar, are at large in London right now.

On the surface it is baffling: the tiny island with the mighty reach. Look at other countries of a similar physical size: Qatar, Gambia, the Lebanon. And those with a similar population: Mongolia, Armenia, Kuwait. Why have these nations not produced a culture that transformed the way the entire world makes and listens to music? Why do their athletes not leave those of superpowers such as Russia, China and the USA trailing in their wake? Why are their dialects not the lingua franca of an entire generation of young people? And why, at the bleaker end of the spectrum of notoriety, have their criminals not become among the most feared and infamous in the world?

As an 11-year-old boy, I was drawn by Jamaica – something in the cadences of the name itself that hinted at a world infinitely more exotic and mysterious than my own. And so, long before I heard a reggae record or knew the taste of overproof rum, I became smitten with the island, and wrote a letter containing this exciting news to the High Commission in London. A kindly reply arrived a few weeks later, enclosing a map and some gaudy brochures. By the age of 17, the dope-smoking bad boys at school had turned me on to pirate radio stations such as the Dread Broadcasting Corporation. When I had the chance to go travelling for a year before university, in 1985, I bought a flight to Kingston. My parents maintained a heroic calm.

It has long been noted that there is an affinity between reggae music and the English public schoolboy. The nice chap who likes a "spliff" and knows all the words to the first Ini Kamoze record has become something of a caricature. Perhaps for those whose only struggle is to get out of bed in time for Latin prayers, the struggle to get out of the ghetto seems impossibly exciting – gritty, authentic, real. Yet the improbable marriage of the privileged and the penniless is at the heart of the music's history – indeed of Jamaica's itself.

It is not stretching a point to say that reggae owes its international success – a phenomenon also 50 years old – to the vision and dynamism of one nice chap who likes a spliff and went to Harrow. When I returned to Kingston earlier this year, it was to the home of Island Records founder, former Bob Marley producer Chris Blackwell, that I was headed. Or at least to one of his homes. Since the early 60s, when he drove around London selling the first ska records out of his car, Blackwell has amassed a fortune – first from music and then from Island Outpost, the chain of hotels he built after selling his stake in the record label for $300m in 1989.

The British once built a hospital up at Strawberry Hill for the coolness of the air. It's the site of the first Island Outpost, 3,000ft above Kingston. Zonked from the flight, sated on rum punch, snapper and rice, our little group gazed down from the restaurant at the great port in the inky dark – 400 years of Caribbean history glittering below us: the unimaginable wealth of the sugar trade; the unimaginable savagery of trading slaves. And then Grace Jones exploded on to the terrace in her knickers. She was with Blackwell, and screeching about a trip to Brazil, a missed flight, a confrontation at the airport. Of the knickers, she offered no explanation. There were vague introductions, but mostly the Jones histrionics, and the lullaby of Blackwell calming her in a tone that suggested many decades of practice. "Ah, Shaggy!" he said, looking behind us, and now here was that very singer, Jheri curls glinting, just passing by with his wife apparently, and popped in to say, "Wh'appen?" With Jones still spitting her indignation, we stammered a greeting to Mr Boombastic, and indeed Mrs Boombastic, as if we did this kind of thing every day.

At 75, Blackwell has been doing this kind of thing for many years. He proves an elusive presence, a Jay Gatsby-ish figure glimpsed only occasionally at the long party of his own life. His family line in Jamaica dates back to the 16th century, its wealth founded on the sugar boom of the 18th and 19th centuries, when slave labour and the island's astonishing fertility fed the new sweet tooth of the Old World and made fortunes for its English plantation owners. But his life has also spanned the full 50 years of young, independent Jamaica, and while he may be a white Jamaican, a tiny group that still wields considerable power, he has done more than any other man – bar one – to fix in people's minds an image of the island that is not just positive, but almost utopian.

The other man is, of course, Bob Marley, who made Blackwell his millions, but whom he in turn guided, supported, mentored and inspired. "The patron saint of dope-smoking students", was how one of our party described Robert Nesta Marley, as we snaked down the road from our temperate perch for the open-air screening of Macdonald's biopic. That is certainly one view of the singer. Along with the 14 songs on the Legend compilation album, his dreadlocked image – fat one forever at his fingertips – has been a cliché for so long that it is hard to recall a time before they existed.

For the Island Records apparatchiks in our group, 31 years after Marley's death Legend is the "catalogue" title that keeps on giving, still adding millions to every annual balance sheet. Our minibus driver dutifully stuck a copy on, and we rattled into modern-day Kingston in a bizarre heritage bubble, where it was permanently 1977, and the infernal "Three Little Birds" played forever. At each new set of lights a car would pull alongside with its windows down, and drown out the melodies pure and true with the digital bassline of the latest dancehall anthem.

"Don't worry 'bout a thing, cos every little thing gonna be all right": Prince Harry quoted that line on his recent visit to the island. Combined with the music's focus on the raw sound of the bass – a force that hits the listener somewhere visceral, and has gone on to dominate pop music – it is the uplifting, spiritual quality of reggae's lyrics that have given it such far-reaching appeal.

In his 36 years, Bob Marley had 13 children by nine different women, and, at a hotel near Emancipation Park, where the big screen had been set up, there was a press conference featuring various Marleys. In the lobby a noticeboard announced another event on a different floor: The New World of Marketing and Brands. Certainly the link was not lost on the impressive Mayor of Kingston, Angela Brown-Burke, who made a speech describing how the Marley name had helped to make Kingston the cultural capital of the Caribbean. "His songs describe virtually every mood of human emotion, and his lyrics resound up until this very day." Even in the 10,000th playing of those Legend tunes, that did seem to hold true. "Open your eyes and look within: Are you satisfied with the life you're living?" That pithy line from "Exodus" seems fairly apposite for our times.

There is now a Marley "Mellow Mood" all-natural relaxation drink, a line of Marley branded headphones and a range of Marley Blue Mountain coffee beans, these last two the work of the engaging and infectiously smiling Rohan Marley, son of Bob and nightclub dancer Janet Hunt, and father to five children with US soul singer Lauryn Hill. "Jamaica," he tells me, proffering a bag of the Marley beans, "is like a small piece of England." Evidence of this is everywhere, but what's striking is that the England it resembles is not the contemporary one, but a distant land from a period drama. "You're most welcome," say the waitresses at Strawberry Hill. The language is straight out of Downton Abbey. In keeping with this sense of Victorian propriety, many of the country's old guard still feel frankly a bit queasy about Bob Marley – a drug taker, a Rastafarian – being the world's best-known Jamaican.

Bob Marley's close friend and longtime artistic director Neville Garrick was 12 when Britain granted Jamaica its independence. Touring the world with the Wailers from 1975-81, he was at the vanguard of a country moving away from that old Englishness. I asked him what independence had meant to him. "We were very colonial and very British," he said. "As a boy, I had learned to sing "God Save the Queen". But it wasn't a fight for independence like the American revolution, so it was not appreciated as strongly as if blood had been shed for it. It was just a flag-waving independence, because the island had become an appendage to Britain, rather than being profitable, after the end of the sugar industry. We became much more creative when we stopped being a colony – we developed our own music that was descended from slavery. We were fighting to put Jamaica on the map, as a people who could do it for themselves. I wouldn't say Britain abandoned us – we still got some aid. We're still part of the Commonwealth. In every police station you still have the Queen's picture. But they probably could have helped more to make things better."

After the screening we crossed to the north of the island, leaving behind the Marley era and entering another: a place and time that has in its way contributed just as much to the magnetism and mystique of Jamaica. It was at Blackwell's property, Goldeneye, that Ian Fleming, for many years a lover of Blackwell's mother, Blanche, wrote – and indeed set – his early James Bond novels. So lush and beautiful is the coastal setting, with its kaleidoscopic marine life and its soft lagoon, that it is not hard to see why the sybaritic film stars of the 50s and 60s were drawn to it. In recent years, at a time when no one was building new hotels on the island, Blackwell invested heavily in making Goldeneye once again a sumptuous destination. After more distant sightings, and assurances that "Mista Bee soon come", I finally got my audience with him after midday, as he emerged from his "morning" swim in the lagoon, like some wise and wizened salamander. While a light breakfast of bite-sized pawpaw and mango wedges was placed before him, he offered a clear vision of the island's place in the world.

"Jamaica is always relevant. It has been for 500 years, since Columbus. For the whole of Central America and northern South America the best port is Port Royal in Kingston – today and then. Anywhere there is a lot of action draws people, and Jamaica has had that since then. When sugar kicked in it was one of the biggest assets of the British empire; and finally the music, which started in 1962, almost exactly at the point of independence and it is still relevant in the world. Anywhere you go in the world you will hear that country's music – some American, some English, and definitely some Jamaican."

I wondered what he thought about the outlook for the next 50 years. "I travel a lot," he says, "and the perception has been that Jamaica is a dangerous place. But in the last year that has started to change. It is getting over its negative image. People are starting to view Jamaica differently now. The removal of Dudus Coke [Kingston gang lord, recently given a 23-year sentence in the US for drug crimes] has had a dramatic effect. The figures for violent crime have dropped by 40%. Jamaica has so many assets: it is ideally located in the world; the people are great; it has amazing natural beauty. For its size, its achievements are absurdly high. The future, as anywhere, is all down to management."

The time had come to leave the luxurious if rather artificial world of Island Outpost, and explore. I hired a car and headed away from the tourist areas and into the countryside. People commonly flag down cars to get a ride, so via towns called Leeds and Ipswich, Malvern and Devon, I pulled over and transported a whole cast of characters through the lush, unspoilt interior: the portly ladies heading home after a day cutting weeds; the incomprehensible dreadlocks with two yams to sell; the embarrassed bank clerk and his mother; the posse of schoolgirls in hysterical giggles. I liked to ask the men if they played a bit of cricket. "Yuh, maan." So do you bowl or bat? The answer was always the same: "Tcha, maan – me a all-rounda!"

When I first came here in 1985, my favourite Jamaican musician was a dub poet called Mutabaruka. A brilliant mind and a gifted orator, Muta was a Rastafarian hardliner who would deliver terrifying polemics about the evils of the white man, often with his shirt off. At the Marley premiere I had recognised him, now a faintly camp figure in kente robes, and host of The Cutting Edge, a popular radio phone-in on Irie FM. He laughed when I wondered if he had a theory about Jamaican excellence. "We have a long tradition of rebellion and uprisings. Other islands would ship their 'bad' slaves to Jamaica. People ask, why is it that we run so fast? Maybe it's because when we hear the gun we start to run. Jamaicans always runnin' from guns."

He seemed also to have mellowed when I asked if he thought modern Jamaica had anything to thank the British for. "We are able to frame ourselves and survive in the western world because of certain things that were learned from the British," he said. "Most of our people of intellectual ability were trained in the British educational system, and they have been able to use that same knowledge to liberate themselves from the colonial experience. To have a good understanding of freedom and how you can be free, you have to have an insight into how the masters used and manipulated ideas to govern the slaves. When you know that, you are able to see how you can break out of it."

The high seriousness and spirituality of Mutabaruka's recordings are almost unrecognisable in the sounds of modern Jamaica, where dancehall's fixation on sex has turned it into a form almost beyond parody. Yet slack lyrics aside, Jamaican pop music is still consistently the most innovative, daring and at times downright bonkers in the world. For my last night I went to see it in action, at Montego Bay's celebrated Friday club night Pier Pressure. There, my preconceptions about dancehall were turned upside down, and the new Jamaica revealed itself a little more.

Dancehall's detractors say that it demeans women, treats them solely as passive sexual objects. Yet the scene at Pier Pressure seemed to tell a different story. The women ran the place. The strength, the poise, the imperiousness of their dancing – this was the absolute focus of the night. Without them there would not have been a night at all. The men just sort of stood there, drunk on rum and Red Bull, while the girls dismissively performed manoeuvres that defied the laws of mechanics. All the lyrics were sung by men, but if you sing only about women, then without women you have no song. In the dancehall, all the power is female.

Jamaica today has a female prime minister, a female mayor of Kingston and a female minister for youth and culture. "The school drop-out rate for boys is much higher than for girls," says Neville Garrick. "At the University of the West Indies in Kingston there are more women students than men. The men tend to fall off the track from being in the streets, hanging on the gully banks, joining gangs, getting into drugs. So the women have advanced – and not just in politics: there are a lot of major companies here where the CEO is a woman."

The people who have got things done in Jamaica for centuries finally have their hands on real power. The 50th-anniversary posters in the capital read: "A nation on a mission", and despite the poverty the overwhelming feeling I took away was one of ebullience and optimism. Garrick agreed: "The old fight and perseverance is still there. We are a poor country, but a talented one. Every youth works their mannerism to try and be somebody. I always describe us as the biggest small island in the world."

I recalled the words of one wise old lady on a veranda near Black River, when I had told her the story of the cricketers. "We are not a people lacking in self-confidence," she said. "Every Jamaican is a star."

Marley is out on DVD on 20 August. An eight-CD box set, Soundsystem – the Story of Jamaican Music is released on 9 August. David Rodigan's reggae show is on Radio 2, Mondays at 11pm