More than 1 in 3 of San Francisco's nearly 27,000 city workers earned $100,000 or more last year - a number that has been growing steadily for the past decade.

The number of city workers paid at least $100,000 in base salary totaled 6,449 last year. When such extras as overtime are included, the number jumped to 9,487 workers, nearly eight times the number from a decade ago. And that calculation doesn't include the cost of often-generous city benefits such as health care and pensions.

The pay data obtained by The Chronicle show that many of the high earners bolstered their base pay with overtime and "other pay," a category that includes payouts for unused vacation days and extra money for working late-night shifts.

Leading 2009's $100,000 Club was the Police Department's Charles Keohane, a deputy chief who retired midyear.

His total payout was $516,118, city records show, the bulk of which came from cashing out stored-up vacation, sick days and comp time. Several other police employees who changed rank or retired also saw their annual earnings swell.

When asked how he felt about landing in the No. 1 spot, Keohane joked, "Not so good, if it's going to get my name in the paper."

The 36-year SFPD veteran, whose last assignment was head of administration, said much of that pay was taken out in taxes. "I helped reduce the deficit," he said.

The average city worker salary in San Francisco is $93,000 before benefits, according to Deputy City Controller Monique Zmuda. The data take into account everyone from park gardeners and street cleaners to attorneys and technology specialists.

Almost 100 city employees made $200,000 or more in 2009; six bumped past $300,000 when overtime and other cash-outs were included.

Muni chief's base pay

Only one city employee had a base salary topping $300,000. Nathaniel Ford, executive director of the Municipal Transportation Agency, made $332,489.

Mayor Gavin Newsom had a base salary of $250,903 in 2009, which put him 29th on the list of best-paid city employees.

The ballooning number of highly paid workers is driven by several factors, including inflation, a persistent reliance on overtime and generous contracts in a city known for its politically potent unions.

The city also negotiated a deal to give raises to some workers who agreed to pick up a portion of their pension contributions, City Controller Ben Rosenfield said. That arrangement pushed almost 2,000 city employees above the $100,000 mark in recent years, he said.

In years past, the $100,000 Club included large numbers of Muni operators, transit supervisors, firefighters, police officers and sheriff's deputies who padded their paychecks with hundreds of hours of overtime, paid out at a rate of time-and-a-half.

But a 2008 rule capped most employees' overtime to 30 percent of base pay, in effect spreading out overtime opportunities to more employees, Zmuda said. That and other efforts to curtail overtime appear to be working, with payments projected to drop to $139.8 million this fiscal year, down from $142.1 million last year and $167.7 million the year before, according to the controller's office.

In the fiscal year that ended in June 2009, city salaries accounted for $2.5 billion of the $6.6 billion budget. That does not include the cost of benefits.

Faced with a $483 million deficit heading into the new fiscal year that starts July 1, Newsom is looking to reduce labor costs to help plug the gap.

The mayor last week struck a tentative deal with union negotiators that includes 12 furlough days, equivalent to a 4.6 percent pay cut, for about 90 percent of city workers. The mayor also has asked city department heads and their top managers to take a 10 percent pay cut. The mayor and his chief of staff plan to give up 15 percent of their salary.

Good work

For some government jobs, San Francisco pays similar salaries as other Bay Area jurisdictions, according to compensation surveys compiled by the city's Department of Human Resources.

For example, a stationary engineer in San Francisco earns a maximum annual base salary of $72,488, 1.6 percent above the Bay Area average. A San Francisco police officer has a maximum base salary of $102,648, 2.8 percent above the average.

Payroll clerks, however, have a base salary of $54,314, 16.4 percent higher than the average, and firefighters make $100,646 in base salary, 6.6 percent above the average, the surveys showed.

San Francisco Chamber of Commerce President and CEO Steve Falk said that annual salaries of $100,000 or more aren't as eye-popping as they were a decade ago.

"I don't think the $100,000 pay threshold continues to be the best measure of city government efficiency. It is certainly a good tool for monitoring anomalies and excessive overtime pay, but the real concern is the pay comparison with the private sector," he said.

Above private sector

He pointed to state Employment Development Department data that show city workers on average earn 20 percent more than those in the private sector in San Francisco.

In addition, Falk said, city workers "have significantly better health and pension benefits" that continue to be the biggest cost driver threatening city services. That needs to be reformed, he said.

Many of the highest-earning city employees - including engineers, doctors and attorneys - are paid comparably to their counterparts in the private sector or can help make up for earning less pay with better benefits.

"City government is becomingly increasingly technical and more sophisticated, and you have to pay for the talent," said Bob Muscat, head of Professional and Technical Engineers Local 21.

In 2007, when Newsom was running for a second term as mayor, he negotiated a 23 percent pay increase for police and firefighters spread over four years. Muni drivers, whose pay is guaranteed under the voter-approved city charter to be the second highest among American transit operators, are set to get an 8 percent raise July 1.

Biggest earners: Search a database of San Francisco's $100,000 Club members at sfg.ly/a7WZxP.