Port Angeles, Washington (CNN) -- On Washington state's remote and wooded Olympic Peninsula, major commotion is usually limited to a log tumbling off an overloaded lumber truck.

But lately the peninsula has been roiled by a noisy debate over the expansion of a Border Patrol station in Port Angeles, a three-hour car and ferry ride away from the U.S.-Canadian land border.

The U.S. Border Patrol is spending nearly $6 million to renovate a Port Angeles building that could house up to 50 of its agents.

Prior to the attacks of September 11, 2001, four agents were stationed in Port Angeles, a city of about 20,000 people some 15 miles across the Strait of Juan de Fuca from Canada.

"It's not needed, there's nothing for them to do up here," said Lois Danks, a local writer and organizer of Stop the Checkpoints, which last month staged a small protest near where the Border Patrol's new station is being built.

She says border agents "drive around and hassle people without any reasonable suspicion of anything except for possibly the color of their skin."

"They park across the street from Hispanic grocery stores and taco stands and watch who comes and goes," according to Danks.

Border Patrol officials deny they target any specific community and say that beyond enforcing immigration laws, they guard the peninsula from drug smuggling and terrorist threats.

Whistle-blower's accusations

In 1999, Ahmed Ressam was stopped by a customs officer at the Port Angeles ferry crossing trying to bring explosives into the country from Canada. Ressam was later convicted of terrorism charges.

Border Patrol officials say most people who live in Port Angeles and the small towns that dot the peninsula support their efforts.

But recent criticism that further ignited the debate came from an unexpected quarter: one of the station's own agents.

"There's nothing to do," Border Patrol agent Christian Sanchez said during a July event in Washington on government whistle-blowers. "There are no gangs or cross-border activity. I haven't seen it."

Sanchez told the Advisory Committee on Transparency, a forum funded by the not-for-profit Sunlight Foundation, he never intended to become a whistle-blower, but decided to speak out publicly after he felt his complaints about the Port Angeles station's "lack of mission" were being brushed aside by supervisors.

Sanchez told the panel he ran afoul of supervisors for refusing overtime he didn't feel he was entitled to since, he said, there was so little work to do.

"The taxpayers are paying us all this extra money to do nothing on this peninsula, where it's a water-based border," Sanchez said during the panel discussion. "It's a burden on the taxpayers right now especially with the economy, with Medicare being cut, with the foreclosures."

Through his attorney, Sanchez turned down CNN's requests for an interview.

His attorney, Tom Devine of the Government Accountability Project, which specializes in whistle-blower cases, said Sanchez still works at the Port Angeles station but has requested a transfer back to the U.S. border with Mexico, where he had previously patrolled.

Devine said Sanchez feared more reprisals like the kind that he said took place after he began criticizing the Port Angeles station.

"Retaliation has increased," Sanchez told the panel on whistle-blowers. "My family has been terrorized, vehicles have been driving by, my mail has been opened."

Henry Rolon, the deputy chief of the Border Patrol sector that oversees the Port Angles station, said he was unable to comment on Sanchez's case due to an ongoing investigation.

But Rolon rejected Sanchez's statements that Port Angeles agents are "bored" and "without a mission."

"Agents in Port Angeles have a very important mission and there's lots to do," Rolon said. "You have to go out there, you have to patrol within the community, on the border. Otherwise you are not going to be there when an incident occurs."

It's not clear how many incidents are handled specifically by Port Angeles agents, since the agency does not release statistics for individual stations, according to Border Patrol spokesman Rhett Bowlden.

But last year, the Blaine Sector -- which includes the Port Angeles station and four major land border crossings -- apprehended 673 people and confiscated 1,897 pounds of marijuana, 270 pounds of Ecstasy, 3 pounds of cocaine, and 1 ounce of heroin, Bowlden said. There are currently 327 agents stationed in the sector, including an estimated 40 at Port Angeles.

Clallam County Sheriff Bill Benedict said he sympathized with the Port Angeles border agents because they didn't have enough to do.

"I know (the Port Angeles section's) activity. I think they made less than 20 arrests last year," Benedict said during a May community meeting, the Peninsula Daily News newspaper reported.

"I feel a little sorry for the Border Patrol because it is a very lonely, boring job."

Michael Cox, head of the Border Patrol agents' union, rejected that position -- pointing out that "it's a different kind of work environment" from many other jobs.

"You've got to investigate, you've got to use your brains," said Cox, president of the Northwest Region for the National Border Council. "We have hundreds of miles to protect."

Going on patrol

Port Angeles' supervisory agent Jose Romero was eager to show some of those miles of territory during a recent five-hour "ride-along" given to CNN.

The tour started in the Border Patrol's current headquarters, the cramped basement of the downtown Port Angeles federal building.

"It's a little tight in here," Romero said, walking through the warren of empty cubicles with papers stacked high on the desks. If Sanchez was in the office that day, he was nowhere to be seen.

Outside, Romero climbed into an unmarked SUV and headed out onto the peninsula's one-lane roads.

As he drove, Romero pointed out paths leading to marinas, unmarked "logging trails" and small airports.

All were potential smuggling hotspots, Romero explained.

Being a Border Patrol agent on the peninsula involves coordinating with a mishmash of local and Native American tribal police forces, he said.

The Border Patrol's work on the peninsula sometimes takes on aspects of local police work, according to Romero. Agents often lend their search dogs to police operations and respond to car accidents or when huge logs come flying off timber-hauling rigs, he said.

As the unmarked SUV cut through the thick fog in a wooded area, Romero asked, jokingly, "You're not scared of vampires?"

He was referring to the nearby logging town of Forks, the setting for the "Twilight" vampire series.

Pulling into a small marina, Romero again turned serious.

Ten miles in front of us -- through the haze -- was the Canadian coastline. A few vacationers kayaking in bright orange life vests clashed against the deep blue waters.

"Can somebody land here?" Romero asked. "Very possible. Somebody lands in a Zodiac-type boat, watercraft, Jet Ski, they hike it up the road or have a vehicle waiting for them, load it up -- whatever contraband it is, human, narcotics," the Border Patrol agent said.

"And just like that, they are gone."