(CNN) Outgoing Speaker Paul Ryan said Wednesday that under his watch, the US House of Representatives "made a great and lasting difference in the trajectory of this country" in his farewell address at the Library of Congress, a final capstone in a series of events marking his departure after more than two decades on Capitol Hill.

"We have taken on some of the biggest challenges of our time, and we have made a great and lasting difference in the trajectory of this country," Ryan said, saying the chamber has been "the most productive we have had in at least a generation."

In his remarks, Ryan also acknowledged both the difficulties of the state of US politics as well as recognized areas he fell short of his own goals.

"I acknowledge plainly that my ambitions for entitlement reform have outpaced the political reality and I consider this our greatest unfinished business," Ryan said.

Ryan also lamented sharp partisanship and toxic political rhetoric, starting "from a place of outrage, and seeks to tear us down from there."

"Too often, genuine disagreement quickly gives way to intense distrust. We spend far more time trying to convict one another than we do developing our own convictions."

Ryan was introduced by another retiring Republican member of the House, Rep. Trey Gowdy of South Carolina, who said, "Paul was my favorite member of Congress before I ever got to Congress." Gowdy also gently ribbed Ryan for his proclivity for exercise: "Where I saw Paul most often was in the gym. How you've worked out so much for eight years and have such little muscles is beyond me."

Wednesday's location at the Library of Congress is the same setting where Ryan delivered a major speech a little more than three years ago , projecting a vision of a "confident America" as he reluctantly started his tenure as speaker of the House.

Why Ryan is leaving town

This year, just months after Republicans passed an overhaul of the US tax code -- a longtime dream for Ryan -- the Wisconsin Republican announced in April that he would be retiring from Congress at the end of his term.

"You realize something when you take this job," Ryan told reporters at the time. "It's a big job with a lot riding on you ... but you also know this is ... a job that does not last forever. ... You realize you hold the office for just a small part of our history. So you better make the most of it."

"In an optimistic and grateful farewell, the speaker will of course reflect on his career and accomplishments but intends to focus on the future and issues on which work must continue," Ryan's spokeswoman AshLee Strong told CNN in a statement. "He will talk about his faith in our system of government and say that fixing our politics is all that stands in the way of enacting the solutions needed to ensure the next century is our greatest."

Ryan was a staffer on Capitol Hill before he was elected to Congress in 1998 at the age of 28 to represent Wisconsin's 1st Congressional District. He escalated as a rising GOP star and policy wonk, ultimately becoming the chairman of two major committees, the 2012 GOP vice presidential candidate and speaker of the House.

As he prepares to leave Congress, Ryan has been reflecting on his time in Washington in a series of speeches and interviews. His office this week released a six-part video series on what he sees as his lasting legacy: the massive tax package signed into law by President Donald Trump in late 2017.

It was the first time in three decades that Congress had passed an overhaul of the tax system, and Ryan describes in the video series how it was an effort years in the making.

"It was 2007," Ryan says. "House Republicans had just lost the majority, and I was sitting in a deer hunting tree stand one Saturday morning. That's when I decided to go big and put together a completely comprehensive plan to update the nation's entitlement system and reform the tax code."

Reflecting on his career and accomplishments

Ryan says his time as chairman of the House Ways and Means Committee was pivotal in setting the stage for tax restructuring, and a key factor in his hesitation to become speaker three years ago, after then-Speaker John Boehner, an Ohio Republican, announced he was leaving and Republicans urged Ryan to take the job amid a divided GOP conference.

"I realized one of the obstacles that I knew I would have to experience, if I was going to get tax reform done as Ways and Means chair, was reluctance or resistance from leadership," he says in the video. "Well, since I became the speaker, I realized that was never going to be a problem anymore."

While Ryan and other GOP leaders urged members to campaign on the new tax law in this year's midterm elections, some Republicans expressed frustration with the President's focus on immigration in the final days before the election. Republicans went on to lose their majority in the House but expanded it in the Senate.

"So, if we can put the partisan knives down on the immigration issue — which both sides do — we fix that, and we get the debt under control, there's no stopping our country," he said.

Ryan, now 48, has been coy about his post-Washington plans. He's frequently mentioned as a potential candidate for president or governor down the road, but he mostly says he's looking forward to spending time with his family.

"I always like my job, there's never a job I didn't like. So I'm hoping to do some different jobs that are different than what I've done before," he told Kane in the interview.

Ryan, who has Irish ancestry, also opened the door to being an ambassador someday.

"That's the only other government job I would aspire to, in my 60s, to be ambassador of Ireland," he said, when asked if he would consider it.

This story has been updated and will continue to update with additional developments from Wednesday.