Hong Kong’s courageous demonstrators have won a significant victory – but the struggle to halt stealthy, long-term Chinese efforts to curtail legally guaranteed freedoms in Britain’s former colony is by no means over. Their protest campaign has come to symbolise the wider, global confrontation between authoritarian and democratic, law-based forms of governance – and that epic battle, too, has a long way to run. By agreeing on Wednesday to officially withdraw the proposed extradition bill that first sparked the protests in June, Carrie Lam, Hong Kong’s chief executive, has finally done what she probably wanted to do weeks ago – but was prevented from doing by Beijing. In leaked remarks last week, Lam admitted making “unforgivable” mistakes that plunged Hong Kong into an unprecedented post-1997 crisis.

Yet it is clear that her freedom of action was severely limited. Lam would almost certainly have resigned by now, had her Communist party masters let her. But they feared that any attempt to quit would be seen around the world as a sign of their weakness and indecision – and so are insisting, for now at least, that she stay in post. But Lam’s position has been fatally weakened. If and when calm returns, Beijing is expected to replace her.

The many-headed protest movement must now decide whether to continue to pursue its broader aim of deepening and broadening Hong Kong democracy

Any such move is fraught with new difficulties. It could reignite the fury on the streets that all of China’s threats and intimidation, intensified police violence and mass arrests, have so far failed to quell. At present, Hong Kong’s chief executive is elected by a 1,200-member committee dominated by pro-Beijing politicians. A key demand of the demonstrators is that this tame electoral college be replaced by universal suffrage – and not simply be allowed to install a Beijing-approved hardliner as Lam’s successor.

The climbdown on the extradition bill, which would have allowed Hong Kong residents to be tried in government-controlled courts on the mainland, does not satisfy other central demands made during nearly four months of protests. They include an independent inquiry into police violence and an amnesty for the estimated 1,200 people who have been detained. Lam continued to oppose both demands in her Wednesday statement, while offering renewed dialogue.

Having secured its initial objective, the many-headed protest movement must now decide whether to continue to pursue these latter demands – and, crucially, its broader (and most contentious) aim of deepening and broadening Hong Kong democracy. For some, the focus has already shifted to this bigger battle. “Too little, too late,” wrote Joshua Wong, who led the 2014 pro-democracy protests, responding to Lam’s statement. There will be differing views among activists on how far protests should go and for how long. It seems certain the politburo in Beijing will never grant Hong Kong’s public the full democratic rights it denies to people in the rest of China. Is further compromise possible? It seems unlikely.

Giant demonstrations, involving hundreds of thousands of residents in peaceful marches, may now begin to dwindle, even as more militant, minority action continues. Hopes of such a de-escalation probably explain Beijing’s volte-face. Chinese officials had spent weeks rejecting any form of deal, denouncing the protest movement, arresting its most prominent figures, beating up many of its supporters, and claiming, without evidence, that the “black hands” of the US, Britain and Taiwan were fomenting unrest.

So has Xi Jinping, China’s president, suddenly lost his nerve? Hardly. The calculation appears to be that by making a concession at this point, Lam and the Hong Kong authorities will be able to claim they have listened to the people’s demands and responded positively. If large-scale protests continue regardless, they will argue that they, at least, have acted responsibly and in good faith. In allowing this move now, Xi – who will certainly have had the final say – may also have been influenced by latest figures suggesting Hong Kong is on the verge of an economic recession. This comes in the destabilising context of an intractable, escalating US-China trade war that may only be halted if Donald Trump decides it is hurting his re-election chances.

The Hong Kong Way protest shows enchantment is a powerful weapon | Antony Dapiran Read more

More importantly still, from Xi’s political point of view, the restoration of a degree of calm would avert the dreadful prospect of China’s lavish 70th birthday celebrations on 1 October being overshadowed by globally televised scenes of fighting and mayhem on the streets of a major Chinese city. What would that say about Xi’s personal authority and the party’s legitimacy? Maybe this is the key. China has consistently rejected outside expressions of concern as unwelcome meddling and interference. But Xi knows that international scrutiny is inescapable. In this age of instant global news coverage, unavoidable satellite surveillance, ubiquitous social media and limited censorship capabilities, the Chinese state – for all its enormous, autocratic power – could not impose a brutal crackdown, let alone another Tiananmen Square, without facing huge, negative repercussions.

In this respect, and notwithstanding all the unfinished business, the people of Hong Kong scored one for democracy this week – and in doing so, struck a telling blow in the struggle to uphold universal freedoms and human rights.

• Simon Tisdall is a foreign affairs commentator