When I first moved to New York City, I lived on 13th Street between Avenues A and B, a pleasant East Village block that had plenty of trees, a coffee shop, a yoga studio, and a latex boutique for rubber fetishists. Walking down the street, you’d never suspect that in 1995 police had used an armored vehicle and riot gear to overcome a barricade made by squatters and activists protesting evictions.

But New York’s history is riddled with popular uprisings and violent clashes. The map above, designed by Molly Roy and originally published in Rebecca Solnit and Joshua Jelly-Schapiro’s new book, Nonstop Metropolis: A New York City Atlas, shows how walking through the city—especially lower Manhattan—means passing over past sites of political violence, where people took to the streets with their demands.

“However you classify riots, New York City has been good at them, or at least good at having them,” the critic Luc Sante writes in the essay accompanying the map.

The New York Orange Riot of 1871. Frank Leslie’s illustrated newspaper/Public domain

As Sante writes, most of New York’s riots were about race, starting with the Negro Riot of 1712. In the 19th century, these riots were sometimes sparked by opponents of slavery—the Eagle Street Riot of 1801 began with an attempt to free slaves—and sometimes by supporters, as when anti-abolitionists attacked the Bowery Theater in 1834. These uprisings aren’t confined to the past, though: the most recent mark on the map is the Kimani Gray riots in 2013.

Some of New York’s more amusing-seeming riots—the Astor Place Theater Riot, the Piggery War, and the Rocking Chair Riot—were really about class. During the Astor Place Theater Riot, working-class theatergoers were rebelling against the performance of a snooty British actor and his upper class supporters. The Piggery War was about eradicating people who depended on raising pigs for a living from increasingly desirable Manhattan real estate.

The Rocking Chairs Riot started during a heat wave, after the city sold an entrepreneur a permit to stock parks with rocking chairs and charge for their use, a ploy meant to discourage “less desirable” people from lingering in the parks.

Straw hats on a more seasonally appropriate day, in July. Library of Congress/LC-DIG-npcc-21981

The Straw Hat Riot of 1922 was mostly about straw hats, though. As all fashionable people knew, straw hats could be worn only until September 15. After that, young men would take great pleasure in knocking stray straw hats of any offending head. But in 1922, the whole tradition got out of control when “mobs of hundreds of boys and young men terrorized whole blocks,” the New York Times reported. They weren’t just snatching straw hats, but grinding them to bits.

One of the strange qualities of riots, though, is that they’re often geographically contained. Some of the riots noted on the map moved, and the dots denote their starting point. But New York is big. Most of these riots would have passed through without affecting most people’s lives. And inevitably, after each incident, the city returned to equilibrium.

The streets that were once full of tattered straw hats or barricaded with furniture became calm once again, and the riots a secret of the past. The city’s busy streets rarely reveal the violence they’ve seen.