Fire has been a symbol of the Olympic games since ancient times. Every four years, before the athletes gather for the games, the emblematic torch makes a long journey to the new host city from the Panathenaic Stadium in Athens—the site of the first modern Olympic games where the tradition began more than a hundred years ago.

Winning the bid to host the Olympics brings honor with considerable cost. Many host cities prepare for the influx of athletes and Olympic-sized crowds by completely overhauling infrastructure and hastily constructing mega-projects, like new sports arenas, that later get demolished or forgotten. Then there’s Athens, every day still using the Panathenaic Stadium, built outside the former city walls on soil that stretches back 2,500 years. This white horseshoe stadium stands triumphantly under the shadow of the Acropolis and remains the world’s only arena made completely from marble.

Some 50,000 people filled the stands of the Panathenaic Stadium for the first Olympics in 1896. Athletes competed for silver medals by cycling, fencing, performing gymnastics, shooting, swimming, weight-lifting, and wrestling. But its history goes back much further.

Socrates and his students escaped the midday sun here under a tree to discuss love and hate near the Ilissos River gently rolling by. Nude male athletes ran track on this site during Athens’ early answer to the Olympics, the Panathenaia celebrations, attracting contestants from all over the Greek empire to a new marble stadium built around 330 B.C. by Athenian orator Lykourgos.

Not unlike a city putting its game face on in preparation for today’s Olympics, such monumental projects instilled pride in the population. The Panathenaic Stadium “deliberately reminded the Athenians—and the rest of the Greek world—of Athenian greatness, when the city dominated the entire Aegean basin,” explained historian Robin Waterfield, who wrote the book chronicling Athens throughout the ages. But shortly after the stadium opened, Alexander the Great died, propelling Greece to unsuccessfully fight Macedonia for independence, then lose the Lamian War. “So you can link the building of the stadium to the end of the great Athenian experiment in democracy,” Waterfield says.

Centuries later, under Rome, Herodes Atticus transformed the stadium into its present horseshoe-shape with soaring stone bleachers. While athletic competitions remained the main purpose, the track also witnessed animal slaughters and gladiators (many of which were criminals) fighting until bloody death. Christianity followed and prevented such pagan rituals, so the Panathenaic Stadium turned into a derelict place used to source marble for nearby buildings.

Related Video: History of the Olympics

At enormous expense shouldered by a generous benefactor, the stadium returned to its former glory when chosen to host the first modern Olympic games in 1896. The Roman-era design remains, this time using Pentelic marble from slopes just outside the city to match the Classical monuments of ancient Athens.