As the flame for the London Olympics is kindled in Greece, Helena Smith looks at the legacy of the 2004 Games

In the soft Attic light, Athens's Olympic sports complex does not look like such a bad place. Men and women jog gently under its great steel arches, athletes go in and out of its giant installations, cyclists race up and down its lanes. Just, one thinks, as it should be, eight years after the birthplace of the Olympics, the city that invented the greatest show on Earth, defied sceptics by holding its own "dream games".

But then you notice little things. The clocks have stopped along the corridors of endless basement offices, outside changing rooms beneath and around the site's velodrome. The light fixtures are rusty; there's tumbleweed almost everywhere and graffiti on the walls. Behind their peeling paint several of the buildings are in decay and locked behind wrought-iron fences, dozens of bigger edifices and hundreds of little office blocks are standing idle.

As the sun dips over the Acropolis and the light fades, the atmosphere, already forlorn, becomes strangely forbidding. Eight years after hosting the Games and as Greece prepares to light the flame ahead of London 2012 in Olympia, Athens's Olympic park, once billed as one of the most complete European athletics complexes, is no testimony to past glories. Instead, it is indicative of misplaced extravagance, desolation and despair.

"They've let the whole place go to pot," says Dimitris Dimitriou, a bank worker escorting his nine-year-old son to a fencing lesson at one of the site's five installations. "The main stadium is a bit better off because it's used by football teams but, if you look around, everything is rotting and rusting. The toilets are filthy, the showers stink and there's no hot water. I don't think anythinghere has been cleaned for years."

Then Dimitriou, who was 28 when Athens opened the Olympics in unexpectedly spectacular style, mutters what every Greek now fears. "And, to think, we are still paying for all this. It's part of the reason why our country is broke. And I hate the thought but he will be paying for it, too," he says, pointing to his son.

No one knows how much the Olympics cost Greece – although many think they played a major role in producing the debt that spurred the country's economic downfall. The estimates vary even though a parliamentary committee is supposed to have announced a figure.

The socialist Pasok party, which oversaw most of the preparations, believes they cost €6bn (£4.8bn). The conservative New Democracy party thinks it is more like €10bn. Then there are people like Sofia Sakorafa, a former Pasok MP and champion javelin thrower who participated in two Olympics, who calculates the cost to be nearer €27bn.

Spyros Kapralos, head of the Hellenic Olympic Committee, says he usually goes for a figure in between. "I wouldn't sign my name to it but I'd say €8bn," he says. "There was a lot of creative accountancy. And a lot of the works were done in such speed and haste they cost a lot more than they should have done. I'm not happy."

OAKA, the Olympic sports complex, is bad, but not as bad as the Helleniko Olympic complex on the other side of town. Here venues are not just ill-maintained but, like the economy, almost in freefall.

Air conditioners and bits of panelling – apparently dislodged by high winds – lie on the ground, while the once gleaming chalet-like apartments for athletes look more like a dilapidated council estate than the manicured compound they once were.

The complex's stadium, also built in 2004 – and used by Greece's national athletics federation – is cracked and crumbling, its seats broken, its stairs smashed, its track ripped.

"The conditions are tragic," says Kyriakos Chondrokoukis as he sits in the stands watching his son, Dimitris, the world record-breaking high jumper, warm up. "A lot of money was spent but there's nothing to show for it. All we've been left with are a lot of empty shells. Go, go, and see for yourself."

He is not wrong. Up the road, world-class venues, purpose-built to host baseball, fencing, hockey, softball, kayak and canoeing events stand eerily empty. But on the edge of the site, facing the sea, another edifice – built to house a restaurant for the thousands of competitors and dignitaries using the facilities – really embodies the profligacy and waste.

Through its dust-covered windows, beyond the litter around it, chairs can be seen, some in their original wrapping, stacked one on top of another; tables are piled high alongside unused kitchen equipment.

"Apparently, it opened for one hour when Costas Simitis, the prime minister, visited before the start of the Games," says the athlete giving us a guided tour. "It never opened again."

There is more. In the working class district of Nikaia, north of the port of Piraeus, the weightlifting centre has almost never been used since the Games, at least for sports purposes. In the northern suburb of Galatsi, a giant 8,000-seat stadium also built in 2004 to host table tennis and rhythmic gymnastics stands empty.

"It's appalling," says Eleni Protonotariou in Galatsi's town hall. "The municipality for years has been trying to open it up for public use so local athletes at least can use it but our efforts have always been thwarted."

Further north, the Olympic village, acknowledged at the time of its construction as the best housing complex ever built for a Games, has the same ghost-like feel. What had been billed as the biggest urban regeneration project in the history of Athens, with a housing capacity for 10,000, is a depressing site, litter-strewn and derelict.

Held in Greece for the first time since their modern reincarnation in Athens in 1896, the Olympics took place under the motto "welcome home". A total of 36 venues were either built or upgraded for the occasion. And the capital duly played up its links to Olympic history – assets that helped it secure the Games – for all it could.

As one of the smallest countries to host the event, the Greeks still speak of 2004 as a defining moment, when the country crackled with optimism, confidence and pride. The defiance of the doomsayers who believed the Olympiad would never get off the ground – given the chaotic countdown to the opening – still elicits cries of delight.

"For a short time we were the centre of the world, people knew that a place called Athens existed," said Dimitris Evangelopoulos, Greece's national track and field coach. "And we pulled off a good Games, everyone says it. "

"But," he adds, warming to the point he really wants to make, " we also lost a lot of money. For such a small country there was a great deal of unnecessary extravagance. For starters, too many [permanent] stadiums were built instead of following the example of Sydney and making temporary installations."

And now, after having bankrupted itself, Greece could not even afford the estimated €60m needed as basic upkeep to operate and maintain the sites. Fed up with the shoddy facilities, Greek Olympians now invariably go to Cyprus to train.

"Now we can't use them or even maintain them," says the national coach. "They're just standing there falling to pieces. It's absolutely scandalous. It's as if the lights went out at the closing ceremony and that was it."

Evangelos Venizelos, the socialist Pasok leader who was in charge of co-ordinating preparations for the Olympics, admits authorities failed to "exploit the Games' success".

"Let's say there was an awkwardness around exploiting Olympic works and the atmosphere the Olympics created," he said.

Kapralos, the Hellenic Olympic Committee's president, does not disagree. "There are white elephants, but the Games did serve to upgrade a big portion of the infrastructure of the city and the country. Greece lives off tourism and after the Olympics, Athens got a new airport, new ring roads, new metro, new tram system, new trolleys, new buses, new telecommunications network, new power stations. The quality of life here improved immensely."

For anyone who lives in Athens that cannot be denied. Yet Kapralos, like politicians across the board, also accepts there was no post-Olympics development plan.

In their rush to get venues completed, Olympic planners overlooked other crucial factors such as environmental strategy and forward thinking. They also failed to draw up economic feasibility studies or even a basic business plan. Very few of the facilities have been successfully exploited commercially.

"No one in Greece is interested in ping pong or kayaking," said Kyriakos Chondrokoukis, a high school physical education teacher when he is not coaching his son. "Greece didn't need big stadiums [to host those events]. What it needs is neighbourhood stadiums for not more than, say, 3000 people."

And what of Greece's Olympics legacy? "But what do we have?" he retorts. "Nothing! To be an athlete in this country you have to be a mythical hero. It would have been better had they never taken place."