Hong Kong police arrested more than 500 people who refused to leave a street in the city's financial district today, a day after tens of thousands of people joined a massive march to push for democracy, free from China's interference.

The march is an annual affair to mark the anniversary of the day China took over Hong Kong from Britain on June 1, 1997 with the promise to give the city a high degree of autonomy for 50 years.

But there is growing unease among its residents especially the youth that the Western-style civil liberties they've know all their lives are being eroded as the freewheeling capitalist enclave comes under Beijing's hard-line communist rule.

The fears are only going to be heightened following the pre-dawn crackdown by the Hong Kong police, who normally do not have an antagonistic relationship with the people, unlike the security forces in mainland China.

Police said 511 people were arrested for unlawful assembly in the Central business district and preventing police from carrying out their duties.

After warnings failed to dislodge the protesters, the police moved in and forcibly removed them. Some went willingly while others lay down on the street with arms locked, and had to be physically carried away one by one.

The protesters had vowed to stay until 8 am, just before the height of rush hour began, but the police started moving in to evict them at about 3 am.

Those arrested were mostly students who had decided to occupy Chater Road after taking part in yesterday's rally, which police said attracted 98,000 people at its peak.

Organisers said 510,000 people turned out, the highest estimates in a decade. Hong Kong researchers put the number at between 154,000 and 172,000.

Whatever the numbers, the march and the vehemence of opposition is certain to raise the alarm in Beijing, which is vowing to restrict the democracy it has promised Hong Kong.

China's Communist leaders have pledged to allow Hong Kong residents to vote for the leader by 2017. However, they've rejected calls to allow the public to name candidates, insisting instead that they be vetted by a Beijing-friendly committee like the one that has hand-picked all leaders since the handover.

Also, three weeks ago, the Chinese government released a so-called white paper that said Hong Kong's high degree of autonomy is not inherent but is authorised by the central government in Beijing.