It is hard to imagine now, but in early 1989, the heart of Berlin resembled what today could only be described as the set of a cold war spy movie. The abandoned, burned-out Reichstag, the East German border guards’ watchtowers, barbed wire and searchlights and the grim expanse of no-man’s-land were not a piece of make-believe. They were all dreadfully real. And at its centre, the Berlin Wall, dividing a city, a nation and a world into opposing camps.

It was hard to imagine then that, within a few months, this forbidding barrier would tumble down, the border shootings would cease, East Germans would be free to leave what for many had become an open prison, and the cold war, with its threat of nuclear Armageddon and all its other malign ramifications, was finally coming to an end. “Ich bin ein Berliner,” John F Kennedy had famously declared in 1963. Now, suddenly, everyone was.

Phrases such as “turning point” and “historic watershed” are overused in contemporary discourse. But the fall of the wall, 30 years ago this weekend, was the supreme symbol of a truly hopeful global moment. It transformed Europe’s future prospects. It signalled an end to proxy wars from Luanda and Kabul to Managua and the Ogaden. It foreshadowed and hastened the Soviet Union’s collapse. It elevated the US as sole superpower, overseeing what George HW Bush portentously declared to be a “new world order”. And it brought hope of peace.

From this pivotal moment, much has flowed and continues to flow, in positive and negative vein. What was good and bad about this dramatic metamorphosis, what was useful and what dangerous, are obscure and complex historical questions. But some post-1989 consequences are already plain to see. They form the inescapable context and backdrop to modern life. They are at the root of challenges that beset us today.

The fall of the wall led rapidly to the reunification of Germany and its re-emergence as Europe’s leading political and economic power. Many in former East Germany regretted their country’s eclipse while enjoying the enhanced prosperity and security it entailed. They still do. Many in France and Margaret Thatcher’s Britain, jealous of their national power, also fretted over this German renaissance, as did weak states with long memories such as Greece. These mixed feelings persist, conditioning and sometimes hindering how present-day Europe functions.

Germany’s resurgence, coupled with the emancipation of former Warsaw Pact satellites, injected enormous momentum into the slumbering European project. Four years after the wall fell, the Maastricht treaty created the EU. Within little more than a decade, enlargement had brought 13 additional states into the fold, several from eastern and central Europe. Europe’s ambition to act as a united, powerful, integrated player on the global stage was reborn.

This aim was not wholly unrealistic. For most Europeans, the post-1989 story was one of relative economic improvement. In Poland, Slovakia and the Baltic states, for example, average incomes doubled and health and life expectancy improved. But far-reaching, disruptive social change, growing inequalities and post-Maastricht free-market capitalism brought their own problems. In time, new European fault lines emerged, running north-south as well as east-west.

The cold war held nationalist instincts in check. But lately, lacking the glue of a common enemy, challenged by post-2008 austerity and roiled by concerns about immigration and identity, Europe’s politics have taken a populist, inward-looking and divisive turn. While most Europeans continue to support EU membership, and notwithstanding the boosterism of France’s president, Emmanuel Macron, confidence in Europe-wide solutions has dwindled. Brexit has both arisen from, and aggravated, these trends.

If the idea of the EU as leading global player is also slipping, that is due, in part, to post-1989 US hegemony. The end of the cold war provided surprising insights into the American psyche. There was much unwarranted triumphalism. There was a largely illusory “peace dividend”. And there was a conviction that the US had the right to act however it wished in the world. This delusion, still not entirely dispelled, culminated in the twin disasters of the “global war on terror” and Iraq. Yet Europe, for the most part, marched to the beat of America’s drum.

In reaction to its reverses and the concomitant rise of its own nationalist-populist movements, the US has since begun to draw back internationally. But far from affording Europe an opportunity, US disengagement, notably ambivalence over Nato, has intensified its insecurities. A reassertive Russia once again looms as a threat in the east. And then there’s the conundrum of China, with which the Trump administration seems hellbent on starting a second cold war.

It would be too much to say the hopes and dreams of 1989 have died. But much of the promise of those heady days has faded. Macron aside, Europe’s current leaders (and Britain’s, too) appear weak, unimaginative and blind to history’s legacy. Despite the divisions, the cold war brought a kind of unity of purpose and belief – and a shared, positive vision of future possibilities. It is that which has been lost and must be found again.