WASHINGTON (CNN) -- Chief Justice of the United States John Roberts renewed his call for higher judicial pay on Wednesday, warning of long-term damage to the fabric of the courts.

Chief Justice John Roberts says federal judges were denied cost-of-living allowances.

In his annual year-end report, Roberts noted judges' salaries have declined over the past two decades when measured against inflation, causing many judges to leave the bench for more lucrative jobs in the private sector.

Roberts, 53, and other judges have said a crisis in the administration of justice could result if the bench cannot attract "judges who are the best of the best."

"I must renew the judiciary's modest petition: Simply provide cost-of-living increases that have been unfairly denied," wrote Roberts. "We have done our part -- it is long past time for Congress to do its."

The last time judges received a substantial pay increase was 1991, but they have received periodic increases designed to keep pace with inflation.

The chief justice noted his colleagues again were denied a cost-of-living allowance this year. Majority Leader Sen. Harry Reid, D-Nevada, insisted the increase be included in the auto bailout legislation considered in December, but it was dropped after some lawmakers objected.

Roberts complained that every federal employee, including all members of Congress, received a cost-of-living allowance this year, except for judges. Lawmakers' cost-of-living allowance increase was 2.8 percent, or $4,700. Congressional salaries are roughly on par with those of district court judges.

House and Senate committees earlier this year approved a 30-percent pay increase for the federal judiciary, but Congress as a whole has not acted on the proposal.

Some members of Congress are cool to the idea of higher pay for federal judges, who are among the highest paid federal employees. They also enjoy lifetime job security, and can retire at 65 with full salary after 15 years on the job.

Sen. Patrick Leahy, D-Vermont and the chairman of the Senate Judiciary Committee, said the judges were "rightfully dismayed that they were not even accorded a cost-of-living adjustment this year."

"I share their disappointment and will continue working to enact the needed legislation to provide that adjustment," he said.

Leahy lamented that his own bill on judicial compensation, which he said would have made up for the lack of cost-of-living allowances in previous years, failed to pass the House or the Senate, even though both Judiciary Committees reported the bill out.

"As the economic crisis has deepened, it will be harder to enact this measure, but these adjustments are no less important," he said.

Higher pay has long been a crusade for Roberts and his predecessor, William Rehnquist. Recent year-end reports have contained more pointed criticism of congressional inaction, but Roberts' comments this year were relatively muted. As chief justice, he has administrative oversight over the entire federal court system. He is also the highest paid federal judge.

"The legal issues in today's global, technology-driven economy are increasingly complex, and judges must respond with wisdom and skill acquired from study, reflection, and experience," said Roberts. "The government must attract and retain the finest legal minds, including accomplished lawyers who are already in high demand, to join the bench as a lifelong calling."

Roberts proudly noted belt tightening in the federal courts, which submitted a budget calling for no new spending. The third branch of government only makes up two-tenths of one percent of the $3 trillion annual U.S. budget. He called that "miniscule."

All About John Roberts (Chief Justice) • Judiciary • Patrick Leahy • Harry Reid