Story highlights Beverly Hills City Council votes 5-0 in telling Brunei to get out of town

Council condemns Brunei's "extreme and inhumane" Sharia laws

City wants Asian country to sell off its interest in landmark Beverly Hills Hotel

But hotel workers pack council chambers and say they fear for their jobs

The Beverly Hills City Council has condemned the government of Brunei for its "extreme and inhumane" Sharia laws and urged the nation's sultan to sell ownership of the landmark Beverly Hills Hotel and any other property the Asian country and its ruler may own in the exclusive community.

The council made the declaration in a 5-0 vote Tuesday night, despite 125 workers from the hotel packing the council chambers and expressing concern about their jobs.

Several waiters in white coats told the council they, too, didn't like Brunei's laws but they asked why workers should possibly suffer for the City Council's vote of condemnation.

"It strangles our livelihood," said Ann Romer, a server in the hotel's Polo Lounge, a renowned gathering spot for Hollywood stars and filmmakers. "It causes us to be unable to support our families, my children and my sick grandmother in Vietnam."

All the council members empathized with the workers but said they need to take a stand against the Brunei government.

"The evil is clear. The evil exists in the sultan of Brunei and others who believe as he does," said Vice Mayor Julian A. Gold, who's also an anesthesiologist. "We have 600 people who work at this hotel who are truly innocent in this and who could be collateral damage."

JUST WATCHED Leno wife on Beverly Hills Hotel boycott Replay More Videos ... MUST WATCH Leno wife on Beverly Hills Hotel boycott 04:14

JUST WATCHED Author: Sultan of Brunei breaks own rules Replay More Videos ... MUST WATCH Author: Sultan of Brunei breaks own rules 02:30

The council vote came a day after Jay Leno and his wife, Mavis, led several celebrities in a demonstration across from the famed Sunset Boulevard hotel. They protested the hotel's ownership, which is an investment group controlled by the Sultan of Brunei.

In late April, the small Muslim nation on the northern coast of the island of Borneo implemented Sharia law, marked by strong punishments for those found guilty of homosexuality or adultery -- including stoning to death those convicted.

The law was praised by the sultan, who isn't just the country's ruler but also a major political and cultural figure.

"We're just making people aware," Leno told CNN. "It's not a political issue. This is not something that's debatable. ... It's people being stoned to death."

In the wake of the protest movement, a number of organizations have moved -- or threatened to move -- events scheduled for the hotel.

The resolution approved by the Beverly Hills City Council accuses Brunei of imposing harsh penalties for homosexuality and adultery. That country's recent laws also allow flogging and severing limbs for theft and other offenses, the council asserted.

"The City of Beverly Hills urges the government of Brunei to divest itself of the Beverly Hills Hotel and any other properties it may own in Beverly Hills," the approved resolution said.

Other celebrities including Ellen DeGeneres and mogul Richard Branson said they won't patronize the hotel or other properties in the Dorchester Collection of luxury hotels, which owns and manages the Beverly Hills Hotel and the Hotel Bel-Air.

Leno said he was well aware of how "a number of people are canceling events at the hotel, and it's all economics," he told CNN.

But Christopher Cowdray, the Dorchester Collection's CEO, said the celebrities and groups protesting are taking out their anger on the wrong place.

"The actions they're taking are unfounded," he said.

"It's going to hurt our employees and this has nothing to do with them whatsoever," he said. "It's not their fight."

The sultan has owned the Beverly Hills Hotel since 1987. The Dorchester Collection was established in 1996 to oversee that hotel and others in its luxury portfolio, including the Hotel Bel-Air, Paris' Plaza Athenee and London's Dorchester.

Cowdray said that celebrities wanting to influence politics in Brunei would be better off lobbying the U.S. State Department to take action.

At a briefing Monday, State Department spokeswoman Marie Harf said, "We have very serious concerns."

The Beverly Hills Hotel has been a haven for Hollywood's elite since 1912 -- two years before the City of Beverly Hills was even created.