Burns, Oregon (CNN) The armed occupiers who took over a federal wildlife refuge in Oregon will soon reveal their departure date, one of the group's leaders said Tuesday at a press conference.

"Friday at seven, we will be going into town and holding a meeting with the community to explain to the community ... why we are here and when we will be leaving," LaVoy Finicum said.

He didn't say exactly where the meeting would be held in the nearby town of Burns. Ammon Bundy, the group leader who usually addresses the media, will attend, he said.

The group has occupied the Malheur National Wildlife Refuge in southeast Oregon for 11 days to protest federal land policies.

On Monday, the occupiers used government equipment to uproot and tear down a fence at the request of a local rancher, CNN affiliate KTVZ reported. They said it was part of the group's bigger plan to return federal land to residents of Harney County.

The U.S. Fish and Wildlife Service issued a statement Tuesday, saying: "Removing fences, damaging any Refuge property, or unauthorized use of equipment would be additional unlawful actions by the illegal occupiers. Any movement of cattle onto the Refuge or other activities that are not specifically authorized by USFWS constitutes trespassing.

"If they take down the fences, it hurts the Refuge, but it also destroys the positive conservation impacts reaped from decades of direct collaboration and sweat equity paid by the Harney County (and surrounding) communities, ranchers, landowners, partners and friends."

Sheriff: Police and civilians being harassed

Finicum told reporters the fence was making a local ranch "not profitable" and "hard to operate." He said that before the Bureau of Land Management installed the fence last year, local ranchers had enjoyed access to the range "for many years."

A resident in Burns, Oregon, expresses frustration over the occupation.

Harney County Sheriff Dave Ward said there's been an increase in "vandalism, harassment and intimidation reports" since the group took over the wildlife refuge.

Some of the occupiers have been able to leave and return to the refuge center. And protest leaders have also called on supporters near and far to come join their cause.

The result, according to Ward: the harassment and intimidation of police and civilians, including workers for the U.S. Fish and Wildlife Service, the agency that runs the refuge.

"There are continual reports of law enforcement officers and community members being followed home; of people sitting in cars outside their homes, observing their movements and those of their families; and of people following them and their families as they move around the community," Ward said Monday.

Critics, supporters of group showing up

"Self-identified militia members" approach them in public, Ward said, trying to start a debate about the federal government.

"The people on the refuge -- and those who they have called to our community -- obviously have no consideration for the wishes or needs of the people of Harney County," the sheriff said. "If they did, they, too, would work to bring this situation to a peaceful close."

JUST WATCHED Guess who owns 28% of U.S. land Replay More Videos ... MUST WATCH Guess who owns 28% of U.S. land 01:42

Bundy, son of controversial Nevada rancher Cliven Bundy , and others started out protesting the sentencing of Dwight Hammond and his son Steven, two ranchers convicted of arson on federal lands in Oregon.

But the march supporting the Hammonds led to the armed occupation of the refuge , with occupiers decrying what they call government overreach when it comes to federal lands.

Other people are coming from afar to protest the protesters, according to KTVZ.

They're angry and they want the armed protesters to get out.

"These are my public lands, these are your public lands, and what I see is a lunatic fringe of radical extremists who have taken my land over," said Garrett VeneKlasen, a protester with the New Mexico Wildlife Federation.

JUST WATCHED Oregon activist: FBI has warrants for our arrest Replay More Videos ... MUST WATCH Oregon activist: FBI has warrants for our arrest 01:42

'They are domestic terrorists'

After Bundy and his group finished a news conference, VeneKlasen took to the same stage to protest the occupation.

More opponents to Bundy's group arrive on the refuge #Oregonstandoff pic.twitter.com/cRUff9db14 — Wanda Moore (@WandaKTVZ) January 11, 2016

"I came here from New Mexico to speak on behalf of sportsmen, to speak on behalf of people that love land and wildlife, to come to tell these people to get the hell off of our land," he said. "Let's not candy-coat this thing -- they are terrorists. They are domestic terrorists."

Another protester, who was holding a sign that said "Get the flock outta my wildlife refuge," told KTVZ, "I never thought I'd have to say this, but I'm here to oppose the armed occupation of a wildlife refuge."

Franki Gould attends a community meeting about the takeover.

At a community meeting of a few hundred people in Burns, Franki Gould told CNN that she wants the protesters gone.

"We were a really close town and now we're very divided," she said.