For news out of east Asia, it is difficult to compete with North Korea’s youthful, jocular despot, Kim Jong-un, and his near-daily threats to fire a nuclear-tipped missile at US territory. On Monday, Kim was pictured surrounded by his top generals mulling over maps with targets closer to home, in South Korea and Japan, while warning again that he was ready to “wring the windpipes of the Yankees”. The young Kim, and his father and grandfather before him, have long tossed violent epithets at their enemies, but Pyongyang’s new capabilities – to potentially deliver a nuclear warhead across the Pacific – have injected fresh danger into the crisis on the Korean peninsula.

The North Korean crisis is one of the few creations of the cold war to have outlived the Berlin Wall, despite persistent predictions that the communist dynasty would collapse. There are many factors driving the confrontation, chief among them paranoia in Pyongyang, where the Kim dynasty is focused above all on preventing regime change. In neighbouring China, Beijing is paralysed: it is caught between anger at Kim for destabilising the region, and fear that if it pushes Pyongyang too hard, the regime will collapse, and fall into the hands of South Korea, an ally of the US. The US itself also seems impotent, knowing that starting any war could lead to devastating attacks on its allies in Seoul and Tokyo.

Lost among the headlines is the fact that the crisis is just a symptom of a bigger drama unfolding in east Asia, where an entire postwar order, built and maintained by the US since 1945, is slowly coming apart.

While the US military bases still dotted across the region have a whiff of latter-day imperialism, for the past seven decades Pax Americana has underwritten an explosion in wealth not matched in the world since the industrial revolution. Since the 1950s, Japan, and then South Korea, Taiwan and China, have been able to put bitter political and historical enmities aside to pursue economic growth.

At the same time, the US presence in east Asia has papered over serial diplomatic failures. All of the frozen-in-the-1950s conflicts buried during the decades of high-speed economic growth are starting to resurface. China and Taiwan have drifted further apart than ever politically. The Korean peninsula remains divided and bristling with conventional and nuclear armoury. The Sino-Japanese rivalry overflows with bitterness, despite a bilateral business relationship that is one of the most valuable in the world.

The Chinese often quote an ancient idiom when speaking about Japan: two tigers cannot live on one mountain. China is in growing competition with Japan to be the dominant indigenous power in Asia, and many view this as a zero-sum game. Any clash between them would not be a simple spat between neighbours. A single shot fired in anger could trigger a global economic tsunami, engulfing political capitals, trade routes, manufacturing centres and retail outlets on every continent.

Whether these tensions play out peacefully depends not just on the two superpowers, the US and China. Japan – which has at different times threatened to eclipse them both – is also pivotal to regional stability. Prior to Donald Trump’s emergence, it was assumed that just about any scenario for the US in east Asia would involve broad continuity for the core elements of past policy, including trade liberalisation and a commitment to alliances, such as that with Japan. But Trump is also a living embodiment of the larger trend that the days of US dominance of the region are numbered.

Facebook Twitter Pinterest Japanese prime minister Shinzo Abe with US president Donald Trump at the White House in February. Photograph: Jim Bourg/Reuters

Today the relationship between the three powers resembles a geopolitical version of the scene in the movie Reservoir Dogs in which a trio of antagonists simultaneously point guns at each other, creating a circle of cascading threats. In the east Asian version of this scenario, the US has its arsenal trained on China; China, in turn, menaces Japan and the US; in ways that are rarely noticed, Japan completes the triangle, with its hold over the US. If Tokyo were to lose faith in Washington and downgrade its alliance or trigger a conflict with Beijing, the effect would be the same: to overturn the postwar system. In this trilateral game of chicken, only one of the parties needs to fire its weapons for all three to be thrown into war. Put another way: if China is the key to Asia, then Japan is the key to China, and the US the key to Japan.

In recent months, it has become fashionable among American journalists and foreign policy analysts to warn of the so-called Thucydides trap – the idea that a rising power (China, in this case) is destined to go to war with an established power (the US). But there is another geo-strategic dilemma identified by the same ancient Greek historian, which is more pertinent. It is dangerous to build an empire, Thucydides warned; it is even more dangerous to give it away.

This “other Thucydides trap” encapsulates the real dilemma faced by the US in east Asia. After more than seven decades as the region’s hegemon, the US now has a choice to make. It could stand and fight to maintain the status quo, at potentially massive cost. Or it could retreat from east Asia, potentially leaving a trail of chaos in its wake.

During the presidential campaign, Trump suggested that Japan and South Korea had become over-reliant on US security, and that it was time for the US to pack and up and go home. But Asia’s economic rise has only magnified the dangers of an American drawdown. “It is not only true that China changed the status quo by getting strong,” said Yan Xuetong, one of China’s most prominent hawks, “but also that America and Japan changed the status quo by getting weak.”

A Chinese friend, trying to describe how Washington views east Asia, came up with a disarmingly simple formula. “The Americans like the Chinese, but they don’t like China,” he said, and then: “They like Japan, but don’t like the Japanese.” George Kennan, the renowned strategist, called Japan’s partnership with the US “an unnatural intimacy”, born of conflict between two very different countries, which, over time, developed into a close relationship of its own. This intimacy – if that is what it is – has been hard won. A remarkable number of senior US officials, starting with Henry Kissinger, have not hidden their dislike for dealing with Tokyo. In his authorised biography, Brent Scowcroft, a hard-nosed veteran of America’s national security establishment, called Japan “probably the most difficult country” the US had to deal with: “I don’t think we understood the Japanese and I don’t think the Japanese understood us.”

Japan to develop missile as tensions with China mount – reports Read more

It is not only the Americans who feel uneasy about the relationship. Washington originally saw the alliance as a way to ensure that Japan was on its side in the cold war and, later, that it stayed in sync with the US’s broader global strategy. By contrast, for Tokyo, according to the Japan scholar Kenneth Pyle, the security pact was an “unpleasant reality” imposed on the nation after the war, but one it cleverly and cynically made the best of. All the while, Tokyo has harboured the fear that the US and China are natural partners – big, boisterous continental economies and military superpowers that wouldn’t hesitate to bypass Tokyo in a flash, if only they could find a way to do so.

Into this volatile landscape strode Donald Trump, Republican candidate and now president, a man who cut his teeth politically in the 1980s with attacks on Japanese trade practices. On the campaign trail, Trump criticised Japan and South Korea for free-riding on US military power, and said both countries should acquire nuclear weapons if they wished to reduce their reliance on Washington. On trade, he singled out China and Japan for cheating Americans, in league with the domestic Visigoths of globalisation, Wall Street and big business.

In the White House, Trump has slightly altered his rhetoric, paying lip service to the conventions of the postwar order. When Shinzo Abe, Japan’s prime minister, visited soon after the election, Trump repeated a commitment made by his predecessors, saying that the two countries’ bilateral defence treaty covered the Japan’s Senkaku Islands, which are also claimed by the Chinese, as the Diaoyu. Diplomats in Washington told me after the meeting that Trump had only done this after being talked into it by his daughter, Ivanka, who had been lobbied by the Japanese.

Facebook Twitter Pinterest The Senkaku Islands, owned by Japan but also claimed (as the Diaoyu Islands) by China. Photograph: Kyodo/Reuters

Even if Trump accepts that the US, for the moment, has to abide by its treaty obligations to Japan, and other regional allies, he has never made the argument, during the campaign or in office, as to why it should. On the question of the other “Thucydides trap” – the principle that it is dangerous to build an empire but more dangerous to let it go – Trump had seemed quite unconcerned; that was something for other countries to worry about. Far from fretting about Japan’s ability to defend itself against China, Trump seemed to believe it would do fine.

In an interview with the Economist in September 2015, Trump was asked what would happen if China started bullying its neighbours without the US being there to protect them. He cast his mind back to more than a century earlier, when Japan and China began to fall into conflict. “If we step back, they will protect themselves very well,” Trump said. “Remember when Japan used to beat China routinely in wars? You know that, right? Japan used to beat China, they routinely beat China. Why are we defending them at all?”

Trump, in his clumsy way, had hit on an existential point, one that he exploited brilliantly in his campaign. Why do Asian countries need the US in the region anyway? Why can’t they get on with each other independent of the US? To fully grasp this dilemma, it is essential to understand the poisonous relationship between China and Japan.

Most accounts of Sino-Japanese relations paint the two countries’ differences as the inevitable result of Japan’s invasion and occupation of China in the 1930s, and throughout the second world war, until Tokyo’s surrender in August 1945, followed by an extended squabble over responsibility for the conflict. Alternatively, their clash is depicted as a traditional great-power contest, with an ascending superpower, China, running up against a now-weaker rival. A third template takes a longer view: one of a China bent on rebuilding the influence it enjoyed in Asia in imperial times.

China accuses Japan of threatening Pacific peace with military law Read more

None of these templates alone, however, captures the tangled emotions and complex psychology of the Sino-Japanese relationship. For centuries, China had been both the Athens and the Rome of east Asia, an empire that established a template of cultural, political and institutional values and structures that permeated the region. Japan’s scripts, its merit-based bureaucracy, hierarchical social relations and exam-intense education system – all of which remain embedded in the country’s 21st-century way of life and governing institutions – originated in China.

In small, striking ways, the Japanese display an authentic affinity with their Chinese heritage. In early 2016, at a farewell reception for a senior Japanese diplomat in Washington, each guest, including the Chinese ambassador, was given a copy of a poem as he or she departed. Penned by the diplomat in whose honour the reception was held, the poem – which celebrated the seasonal blooming of cherry blossoms in Washington – was written in Chinese characters in the style of revered Tang dynasty poets. The gift was an homage to the enduring influence of Chinese culture and to contemporary education in Japan, where schoolchildren still learn the art of classical Chinese poetry.

The histories of modern Japan and China have much in common as well. Both were forcibly opened in the 19th century at the point of a gun wielded by an imperialist west. In the century that followed, they both battled to win the respect of the intruders who considered themselves racially superior to Asians. And yet, far from displaying solidarity with each other, the two nations went in different directions: Japan modernised rapidly, while China disintegrated. Ever since, they have struggled to find an equilibrium of their own. If one country was ascendant, the other was subordinate.

Facebook Twitter Pinterest An anti-Japan protest in Hong Kong in 2012. Photograph: Kin Cheung/AP

Despite their shared roots, Japan and China have remained as psychically remote as they are geographically close. In Europe, an acknowledgment of the second world war’s calamities helped bring the continent’s nations together in the aftermath of the conflict. In east Asia, by contrast, the war and its history have never been settled, politically, diplomatically or emotionally. There has been little of the introspection and statesmanship that helped Europe to heal its wounds.

A corrosive mutual antipathy has gradually become embedded within Japan and China’s ruling parties, and in large sections of the public. In turn, seemingly unavoidable political divisions – partly driven by constant demands from China for Japan to apologise for its wartime conduct and Japanese hostility to such pressure – have eroded trust and strengthened hyper-nationalists in both countries.

China’s economic rise and Japan’s relative decline have only reinforced this trend. In both capitals, the domestic tail now wags the diplomatic dog as often as the other way around. What once seemed impossible and then merely unlikely is no longer unimaginable: that China and Japan could, within the coming decades, go to war.

The territorial disputes, the enduring strains of the cold war, and China’s demand for respect and fear of containment all help to explain the region’s diplomatic tensions. So, too, does geopolitics, which is the furnace for Sino-Japanese rivalry. But at the core of their rivalry are the two countries’ wildly varying and persistently manipulated memories of the Sino-Japanese wars in Asia.

Even the most basic of disagreements over history still percolate through day-to-day media coverage in Asia, in baffling and insidious ways. Open a Japanese newspaper in 2017 and you might read of a heated debate about whether Japan invaded China – something that is only an issue because conservative Japanese still insist that their country was fighting a war of self-defence in the 1930s and 1940s. Read the state-controlled press in China, and you will see the Communist party drawing legitimacy from its heroic defeat of Japan; in truth, Chiang Kai-shek’s Nationalists carried the burden of fighting the invaders, while the Communists mostly preserved their strength in hinterland hideouts. Scant recognition is given to the US, which fought the Japanese for years before ending the conflict with two atomic bombs.

The history of Sino-Japanese relations since the late 19th century, when the two countries first fought a war, has long had a dominant storyline. Japan encroached on Chinese territory, demanding and then taking bits of land here and there, before eventually launching a full-scale invasion and occupation in the 1930s. Tens of millions of Chinese soldiers and civilians died in the conflict. After its defeat and surrender in 1945, so the narrative goes, Tokyo prevaricated endlessly about apologising to China and making good for the damage wrought by its armies.

The first part of the storyline is true. From the late 19th century onward, Japan did set out to dismember China. Although the precise numbers of casualties are still debated, the Nanjing massacre is not an invention, as some prominent Japanese politicians and historians gratingly insist. Japan committed atrocities, used forced labour from its colonies to support the war effort, and oversaw the recruitment of the so-called “comfort women” for brothels for their soldiers.

The history of the history wars, however, is more complex, with many twists and turns that are lost in today’s shrill headlines. When there was much soul-searching in Japan about the war during the 1950s and 1960s, for example, Beijing had no interest in seeking an apology and reparations. Instead, Mao Zedong and his premier, Zhou Enlai, cultivated relations with Japan in an effort to break the US embargo on their country.

In 1961, in a meeting with a Japanese Socialist Party leader, Mao perversely thanked Japan for invading China, because the turmoil created by the Imperial Army had enabled the CCP to come to power. “We would still be in the mountains and not be able to watch Peking Opera in Beijing,” he said. “It was exactly because the Imperial Japanese Army took up more than half of China that there was no way out for the Chinese people. So we woke up and started armed struggle, established many anti-Japanese bases, and created conditions for the War of Liberation. The Japanese monopolistic capitalists and warlords did a ‘good thing’ to us. If a ‘thank you’ is needed, I would actually like to thank the Japanese warlords.”

Facebook Twitter Pinterest Trump with Chinese president Xi Jinping in Florida in April. Photograph: Jim Watson/AFP/Getty Images

Mao often adopted a freewheeling, sardonic style in conversation, which seemed deliberately aimed at putting his interlocutors either at ease or off balance. But his statements brushing off an apology and expressing gratitude to the Japanese for their invasion are embarrassingly discordant in today’s China, and so jarring that they are invariably airbrushed by the CCP these days. The official explanation contends that Mao used sarcasm to underline how Japan’s invasion had “awakened” the Chinese people. Chinese scholars of Japan who have tried to tread a more independent path say the truth is simpler: Mao had no interest in an apology because he genuinely believed that the CCP owed its victory in the civil war to Japan.

Official policy was tailored in conformity with Mao’s views for much of the next three or four decades, even as it grated with many Chinese who retained visceral memories of Japanese atrocities. As one scholar at a government thinktank in Beijing told me last year: “This came from Mao’s mouth. There was no need for any discussion, or for him to consider outside elements such as public opinion or conflicts between past and present policies. His power was absolute.”

By the mid-1980s, when Beijing decided that Japanese remorse should become a permanent fixture of bilateral relations, Tokyo had come to view such demands as little more than self-serving politics. Some Japanese leaders were willing to apologise, just to deprive China of a ready-made issue to beat them over the head with. “We can apologise as much as China wants. It’s free, and very soon China will become tired of asking for apologies,” the former prime minister Noboru Takeshita confided to foreign ministry officials in the early 1990s.

As it turned out, the Chinese never did tire of receiving apologies. They thought they were the country’s due. But Japan did tire of giving them. In the process, history disputes have become a huge obstacle to a genuine postwar settlement.

The rage expressed in China toward Japan these days over history is the tip of a much larger iceberg. Beijing’s core problem is not with the details of the war itself, but with the diplomatic deals that were agreed to settle it. In Washington’s and Tokyo’s eyes, the San Francisco Treaty of 1951 forms the foundation of the east Asian postwar order. The treaty ended the US occupation, reestablished Japan as a sovereign nation, fixed it as a security partner for the US, and gave the country space to rebuild itself into a modern, prosperous nation. The treaty also laid the basis for Japan’s gradual rapprochement with other former wartime foes in south-east Asia and Australia.

Chinese scholars, in lock-step with the country’s political leaders, use a different template for the region, something that is largely overlooked in Washington. Their reference points are the conferences in Cairo in 1943 and in Potsdam in July 1945, at which the so-called Three Great Allies – the US, the UK and the Republic of China – set the terms for Japan’s unconditional surrender. In the process, as Chinese politicians, historians and activists have begun to argue more forcibly in recent years, Japan was consigned to a permanently subordinate role in the region.

Beijing favours Potsdam, because it disarmed Japan, restored the territories Tokyo had seized in the previous century, and confirmed China’s great-power status. It doesn’t recognise San Francisco, because it enshrines the US-Japan security alliance and the American military presence in east Asia. China was represented at Cairo in the form of the then-Nationalist government, but not at San Francisco in any form.

The notion that Japan should sit inert in east Asia, enduring a kind of life sentence as a result of having lost the war, absurd as it is, is given much credence in China, by its top leaders as well as in the popular political culture. As president Xi Jinping told the visiting Pentagon chief Leon Panetta in late 2012, “The international community must not allow Japan to attempt to negate the results of the World Anti-Fascist War, or challenge the postwar international order.” In another sign of this mindset, a pro-nationalist book that became a bestseller in the mid-90s, The China That Can Say No, had a chapter titled In Some Respects, To Do Nothing Is Japan’s Contribution To The World!

Sheltering under America’s nuclear umbrella in the postwar period, Japan has in fact been a constrained power since its defeat in 1945. The Americans, after all, wrote a new “pacifist” constitution for Japan, which said it should only maintain military forces for its own self-defence. At times, Japan, at least in security terms, has seemed to be “inert” and willing to free-ride on the Americans.

But thanks to China and North Korea, those days are over. Shinzo Abe has fashioned a strong national security policy and strengthened the country’s military. While attention was focused on Pyongyang’s nuclear antics in early August, Japan quietly announced that it was studying equipping its military with offensive weapons, such as cruise missiles, to allow it to strike overseas enemies for the first time since the war.

Facebook Twitter Pinterest An infantry unit in Japan’s self-defence force at a ceremony at Camp Asaka in 2016. Photograph: Kim Kyung Hoon/Reuters

Japan presents a particular challenge to China. Militarily, it is not a pushover like other south-east Asian nations Beijing has clashed with recently, such as the Philippines. In 2012, the central government in Tokyo nationalised the Senkaku Islands in order to prevent a far right-wing nationalist politician, Shintaro Ishihara, from buying the islands from their private owners. At that point, Beijing considered trying to take the islands by force. A retired regional leader with good connections in both China and Japan told me that Beijing had studied its options carefully: “They did a number of basic tabletop exercises to work out, if there was a conflict over the islands, whether China could prevail; I had many conversations with Chinese military planners at the time.” In the end, he said, Beijing concluded that the “co-relation of forces was not with them”. Unlike Japan, which has fought naval wars, China has fought only one, in 1894-5, which it lost. The Chinese had made huge strides as a military power, but not so far that they were confident about taking on their old foe.

Perhaps the most salient factor in China’s calculations over the Senkaku Islands was what might happen if it should lose to Japan. In Tokyo, a military loss would be disastrous, of course, and the government would certainly fall. But that would be nothing compared with the hammer blow to China’s national psyche should Japan prevail. “That would be terminal for the CCP,” the former regional leader observed. “Regime change.”

What’s behind Beijing’s drive to control the South China Sea? | Howard W French Read more

Over time, though, China’s capabilities, and its confidence, are likely to outpace those of its neighbour. Japan knows that China is not going away, whereas one day, the US might. China is keen to emphasise to every nation in Asia a single truth: China’s presence is a geopolitical reality in Asia. The US presence, by contrast, is a geopolitical choice, one that China intends to make more and more costly.

If Tokyo continues to feel threatened, and loses faith in the US, the next step is going nuclear. That will be the definitive sign that Pax Americana in Asia is over, and it could come sooner than anyone thinks.

Main illustration by Lee Martin/Guardian Design

Asia’s Reckoning: China, Japan, the US and the Struggle for Global Power by Richard McGregor will be published by Allen Lane on 5 September.

• Follow the Long Read on Twitter at @gdnlongread, or sign up to the long read weekly email here.

