I don't quite know why I'm on a flight to Athens, except that it seems like the right thing to do. I'm flying out on a last minute whim to hang around outside a conference which may, or may not, be happening and to which I've not been invited. None of you has.

You won't have read about it. You won't have seen a guest list, you won't see photographs of it. It isn't happening. It doesn't exist. I'm flying out to Athens for no reason at all. To have a holiday I don't deserve and can't really afford. Maybe catch a little sunstroke, grab some food poisoning, and come home. Pointless.

Unless, of course, the rumours are true. Unless, as a handful of people are saying, this weekend is Bilderberg. The yearly alignment of the distant stars that shape our destiny. A long weekend at a luxury hotel, where the world's elite get to shake hands, clink glasses, fine-tune their global agenda and squabble over who gets the best sun loungers. I'm guessing that Henry Kissinger brings his own, has it helicoptered in and guarded 24/7 by a CIA special ops team.

If it's happening at all, Kissinger will be here. David Rockefeller will be here. Presidents of banks, and chairmen of boards. The Ben Bernankes and Condoleezza Rices of this world. Heads of oil companies, media magnates, the Queen of the Netherlands and Peter Mandelson. Probably Ben Bernanke, possibly David Cameron. Politicians and financiers from all five corners of the globe (don't let them tell you there are four). And me.

I arrived last night, under cover of darkness. I told the cab driver to stop 50 metres from the hotel. He asked why. I couldn't tell him that it was so I could case the entrance for FBI lenses. I simply muttered that I couldn't explain. His eyes lit up. "Aha! I see! I know!" What did he know? And who is that following us? A man in a BMW. Definite spook.

Get a grip.

The driver drops me on a dark corner of the Athenian Riviera, pats me on the shoulder and says: "You want to smoke some dope?" I decline. I need my senses sharp. I scurry into the hotel, glancing into parked cars, looking for vans with mirrored windows. There aren't any. At reception they seem to have lost my booking (the tentacles of Bilderberg reach far!), but eventually I get checked in, go upstairs, unpack, have a shower, go downstairs, step outside, look across the street and realise I've scurried into the wrong hotel. This is who Bilderberg are up against.

An embarrassing hour later, I set out again from the right hotel, determined to find the location where Bilderberg is said to be happening. Get some early photos, maybe see Hillary Clinton arrive. Although I'll settle for Ken Clarke. It's getting late. Joggers are out. FBI? Secret service? Almost certainly. I trudge on determinedly. After about half an hour I realise I turned the wrong way out of my hotel and I am walking up a deserted coastline towards Athens. I go back to bed. Another untroubled night for Bilderberg.

At breakfast, a heavy-set man with hairy forearms sits opposite me and fiddles with his mobile phone. Definite spook. He eats a hard-boiled egg and watches me struggling with my Coco Pops. My first discovery of the day is to find out what happens to Coco Pops when they're left to sit for a decade in a Greek presentation dish. They turn to gravel.

The spook leaves before me. He got what he came for: a photo of me, sneaked on his mobile and wired already to Quantico in Virginia. And a hard-boiled egg.

Outside, it's a beautiful day, the air smells of sun and seashells, and there is no sign of a global cabal meeting anywhere near. I have a wander. From my meagre, third-hand, internet forum sources, I think I know the hotel where Bilderberg is happening: the Astir Palace resort. Further from my hotel than it looked on Google maps. Note to self: always check the scale on the zoom.

A dozen promontories and dusty dead-ends later, and I'm ready to give up. It's too hot. I don't have a sunhat. The world is going to hell and Vouliagmeni is full of litter. What is it with the Greeks and bins? Do they not see them? Do they not believe they exist? Hidden in plain sight … it's the Bilderberg way. It's too hot. I need some water.

Bilderberg's nemesis: journalist Jim Tucker. Photograph: Charlie Skelton

And then, on the pavement ahead, there he was. I recognised him from the videos. The braces, the loose shirt, the grizzle. The tattered leather briefcase, packed with dark secrets. It was the doyen of Bilderberg hunters himself, Jim Tucker. I addressed him.

"Excuse me ... Mr Tucker?"

"Let's go into my hotel and talk."

Tucker is a man in a hurry. He's not getting any younger, and his old enemy Bilderberg is getting stronger.

"Hot enough for you?" I venture.

"Too hot for a fatboy," he growls.

The exchange makes me feel like a resistance fighter exchanging codewords. Assured of my credentials, Tucker gestures me into his hotel lobby. I can't believe my luck. Suddenly I'm not alone, I'm not hallucinating. Bilderberg is here. Where you find Jim Tucker, you know Bilderberg isn't far away. He's a herring gull, telling me there are whales beneath.

Tucker lights a non-filter cigarette, lays his hat upon the table, and settles back into the lobby sofa to talk ...

Charlie Skelton will be filing regular updates from Athens until he is arrested by shadowy figures in dark glasses