A planned rightwing rally in the shadow of the Golden Gate Bridge that was downgraded to a news conference at a small park fizzled out further on Saturday, after San Francisco police swarmed the park and city workers erected a fence around it.

An organizer for the group Patriot Prayer later spoke in suburban Pacifica with a handful of supporters, after city leaders and police repeatedly voiced concerns that they would draw angry counter-protesters and spark violence.

Organizer Joey Gibson denied his group was looking for trouble. He said members had received anonymous threats on social media and feared civic leaders and law enforcement would fail to protect them.

“My hope is to be able to talk to normal citizens without all the extremists,” Gibson, who identifies as Japanese American, said at the news conference. Other speakers included African Americans, a Latino and a Samoan American. Several said they supported Donald Trump and wanted to join with moderates to promote understanding and free speech.

The pivots by the group did not deter more than 1,000 leftwing counter-protesters from descending on Alamo Square park, where they suspected rightwing supporters still might show up.

Park police officers patrol Crissy Field near the Golden Gate Bridge. Photograph: Josh Edelson/AP

“San Francisco as a whole, we are a liberal city and this is not a place for hate or any sort of bigotry of any kind,” said one counter-protester, Bianca Harris. “I think it’s a really powerful message that we’re sending to people who come here to try to spew messages of hate that it’s just not welcome in this city.”

Police closed the park early in the day and looked on in riot gear as the demonstrators gathered around its perimeter waving signs condemning white supremacists and chanting, “Whose streets? Our streets!” Hundreds of others took to the streets in the Castro neighborhood.

Earlier in the week, San Francisco mayor Ed Lee raised concerns that Patriot Prayer would attract hate speech and potential violence. US representative Nancy Pelosi, a fellow Democrat who represents San Francisco, called the planned rally a “white supremacist” event.

Gibson said his group disavows racism and hatred and insisted his gathering would be peaceful. He said on Saturday in a phone interview that he felt San Francisco’s Democratic leaders had shut him down.

“They’re definitely doing a great job of trying to make sure my message doesn’t come out,” he said.

Members of the group ended the news conference abruptly when they heard members of an anti-fascist movement were headed to Pacifica.

The San Francisco Bay Area is considered a cradle for freedom of speech, and police in San Francisco have traditionally given demonstrators a wide berth. Student activism was born during the 1960s free-speech movement at Berkeley, when thousands of students at the university mobilized to demand that the school drop its ban on political activism.

However, the deadly violence in Charlottesville, Virginia on 12 August during a rally of white supremacists led San Francisco police and civil leaders to rethink their response to protests.

When Gibson canceled the Golden Gate rally on Friday, he said his followers would instead attend an anti-Marxist rally on Sunday in nearby Berkeley. A short time later, the organizer of that rally called it off.

Amber Cummings said in a lengthy statement issued via Facebook that she had “grave concerns for the safety of the people attending my event”.

Cummings said the event was planned “to speak out against the political violence happening to people who do not agree” with leftwing ideology, and that the meaning was being lost as rhetoric around the rally escalated.

The leftwing group By Any Means Necessary, which has been involved in violent confrontations, had vowed to shut down the Berkeley rally.