HONG KONG—Protesters smashed wall-size windows and stormed the city’s legislature on Monday, an unprecedented show of force that ratcheted up the worst social turmoil to rock Hong Kong since it returned to Chinese rule 22 years ago.

The mayhem at the government headquarters, which ended in barrages of police tear gas, capped a day in which 550,000 demonstrators marched peacefully through the streets in scorching summer heat, according to organizer estimates.

It...