Washington (CNN) There was a time when Republicans were eager to have Eric Holder gone from the Justice Department. That time isn't now.

GOP lawmakers began calling for the attorney general's resignation as early as 2011, and they even held him in contempt of Congress a year later.

Nowadays the issue keeping Holder in office has nothing to do with him, or Loretta Lynch, the Brooklyn U.S. attorney President Barack Obama picked to succeed him.

Instead it's a partisan fight over an abortion measure that is tucked into legislation aimed to thwart human trafficking. It's a bill that -- absent the abortion add-on -- enjoys overwhelming bipartisan support.

Republicans say they won't vote on Lynch's nomination unless Democrats yield on the unrelated bill. Some Democrats appear to relish that the fight is showing Republicans the minority party still has power.

"These should be my closing days," Holder joked in a speech in recent days. "Given the Senate's delays in scheduling Loretta Lynch's nomination for a vote, it's almost as if the Republicans in Congress have discovered a new fondness for me. Where was all this affection the last six years?"

As he nears the expected end of this tenure, Holder appears to be leaving with negative favorability ratings -- among those who know who he is, according to a new CNN/ORC poll. Overall, 29% of Americans polled have a favorable impression of him, while 35% have an unfavorable view, and 36% still haven't heard of him.

By now, Holder had hoped to be completing his plans for a cross-country driving trip with a friend -- a longtime wish for when he left office, he told CNN in an interview last year.

The fight already has forced lawmakers to postpone hearings for the Justice Department budget, which had been penciled in to begin this week, according to a Justice Department official.

Postponing those hearings is probably not a bad thing given the contentious tone between Republicans and the attorney general in oversight hearings last fall, which everyone thought would be Holder's last appearance on Capitol Hill.

The White House says Holder's stay is indefinite.

A Republican congressional aide says the irony that the current fight is keeping Holder around much longer than GOP lawmakers want has come up in meetings among Republican Senators. But the stalemate continues.

Over the past six years, Republicans often used Holder as a stand-in for the President in fights over everything from national security to gun rights. They made Holder the first sitting cabinet member to be held in contempt of Congress, in a fight over administration records related to a botched gun investigation known as Fast and Furious.

In February, when the Senate Judiciary Committee voted to approve Lynch, Sen. Lindsey Graham told Lynch he was eager to see Holder go.

"Eric Holder is ready to go, and I wish him well," Graham joked.

In a city where the late Marion Barry hung around long enough to become known as mayor for life, Holder is beginning to throw playful jabs at the fact that every day he is climbing the chart for longest-serving attorneys general.

When Lynch's nomination was announced more than 130 days ago, Holder was No. 4. Now he has surpassed Homer Cummings, who served in Franklin Roosevelt's administration, to become the third-longest serving. He has some ways to go before reaching Janet Reno, who served eight years, with Holder as her deputy.

Sen. Patrick Leahy recently excoriated Republicans for delaying what would be a historic first: the first African American woman attorney general. He noted Lynch has already waited longer than the five previous attorneys general, combined. By Monday that wait would become longer than the seven previous leaders of the Justice Department, a Leahy spokeswoman says.

When the stalemate ends, Lynch is expected to win Senate approval by a narrow margin.

The Justice Department held a tearful farewell party for Holder last month, with a surprise performance by Aretha Franklin.

The week after the party, Holder held a meeting with aides and asked for an update on Lynch's nomination, according to a person familiar with the meeting.

No vote was forthcoming that week, came the answer from Peter Kadzik, who heads the Justice Department's office of legislative affairs.

Maybe there's time for a second farewell party, the attorney general mused. This time with Beyonce as the surprise guest.