Lam Wing-kee, the freed bookseller due to lead a massive protest march through the streets of Hong Kong on the anniversary of its return to Chinese sovereignty, pulled out two hours before it was due to start Friday afternoon local time, citing “serious” threats to his safety. The news was announced in a Facebook post by march organizers, the Civil Human Rights Front.

Sixty-one-year-old Lam is one of five associates of Hong Kong–based Mighty Current publishing who have been, or are being, arbitrarily detained in China for their role in distributing books critical of Chinese leaders. Their detention has made world headlines and is widely seen in Hong Kong as the most flagrant breach by Beijing to date of the territory’s autonomy, supposedly guaranteed by China when it resumed control of the former British colony on July 1, 1997.

His withdrawal from today’s march will exacerbate already deep fears in Hong Kong that Beijing no longer respects its freedoms or jurisdiction — in effect ending the “one country, two systems” principle that has governed the city relationship with China during the past 19 years.

As news spread of Lam’s detention, demonstrators continued to mass at the city’s Victoria Park for the march to Harcourt Road, the thoroughfare in front of the Hong Kong government headquarters and also the main site of the 2014 pro-democracy movement dubbed the Umbrella Revolution. Several yellow umbrellas — a source of shade but also now a ubiquitous symbol of protest — dotted the crowd.

“The government only works for mainland China’s benefit, not ours,” S.K. Ng, a 58-year-old insurance salesman holding a sign that declared the current government unfit to lead Hong Kong, told TIME during the march. “I think [Lam] has been forced into silence. The communists are always doing things underground — we have no ability to do anything except walk and raise our voices.”

“I hope the signal that will go out is that ‘one country, two systems’ is at risk,” Emily Lau, a lawmaker from the Democratic Party and prominent pro-democracy advocate, tells TIME. “People are very worried.”

The anniversary of Hong Kong’s return to China has become a day of protest in which all kinds of social grievances are aired — from calls for more LGBT rights to demands for greater protection for migrant workers. This year, however, there has been disagreement over the unifying demand that would unite all marchers. The organizers say their primary goal is the ouster of Hong Kong’s top official — the deeply unpopular Chief Executive Leung Chun-ying, seen as Beijing’s stooge.

Protesters hold up placards as others shout against pro-China supporters (not in picture) during a demonstration in Hong Kong on July 1, 2015 Bobby Yip—Reuters

“We are asking [Leung] to step down … and we hope political reform can be pushed through,” says Jackie Hung, the Civil Human Rights Front’s deputy convener. “He represents only the interests of specific groups of people.”

But many others, particularly the younger generation, say that the march should be used to call for systemic change.

“Freedom and human rights in Hong Kong have been suppressed,” said Denise Ho, the vocally pro-democracy Hong Kong pop star who recently saw a concert canceled by its corporate organizers, the French cosmetic brand Lancôme. Although the company cited “security concerns,” many slammed the cancellation as kowtowing to China. “There’s no situation worse than right now. We need to speak up before we’re completely opposed and our voices are completely erased.”

“I believe that the core message should be targeted at the system itself instead of a person,” Nathan Law, chairman of the recently formed Demosistō Party, which is led by student leaders from the 2014 protests, tells TIME. “I would rather uphold democracy or self-determination,” he adds.

Self-determination is a concept that is acquiring greater and greater currency in Hong Kong. In an essay for TIME on the anniversary of the Umbrella Revolution last year, Law’s party colleague Joshua Wong — also named one of TIME’s Most Influential Teens of 2014 — called it the “only solution” for the city.

Other groups are calling for outright independence. At least three such groups — Youngspiration, Hong Kong Indigenous and the Hong Kong National Party — will not joining the march but are instead organizing their own so-called black-bloc demonstration at the Liaison Office of the Central People’s Government — China’s headquarters in Hong Kong.

“The bookseller incident is directly threatening our safety” Baggio Leung, the head of Youngspiration, tells TIME. “Beijing is breaking the concept of one country, two systems by themselves, they’re voiding contract.”

Last year’s July 1 rally, held in the aftermath of the Umbrella Revolution’s 79-day street occupation that ultimately failed to alter the status quo, was marked by political fatigue. But this year, turnout at the main protest is expected to be high.

“This year, there might be quite a sizeable attendance because of the bookseller case and also genuine fears about Hong Kong autonomy,” says Willy Lam, a professor at the Chinese University of Hong Kong’s Centre for China Studies. “The pro-democracy activists are genuinely afraid.”

Get The Brief. Sign up to receive the top stories you need to know right now. Please enter a valid email address. Sign Up Now Check the box if you do not wish to receive promotional offers via email from TIME. You can unsubscribe at any time. By signing up you are agreeing to our Terms of Use and Privacy Policy . This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply. Thank you! For your security, we've sent a confirmation email to the address you entered. Click the link to confirm your subscription and begin receiving our newsletters. If you don't get the confirmation within 10 minutes, please check your spam folder.

Write to Rishi Iyengar at rishi.iyengar@timeasia.com.