Since the beginning of the 115th Congress, more than 40 Republicans have left or announced they plan to leave the House by the end of the year. A handful of members have resigned amid scandal . Another group is leaving to run for higher office -- for seats in the Senate or to become governors of their home states.

Still, the majority of Republicans leaving Congress this cycle are leaving politics altogether -- a side effect, they say, of the political climate and President Donald Trump, a commander in chief who hails from their own party.

CNN sat down with three outgoing Republicans -- all of whom left for different reasons -- about why they are calling it quits.

Ryan Costello: The reluctant retirement

Ryan Costello, 41, is still new to Congress.

Elected in 2014, the lanky, laid-back congressman is a rare brand in the House GOP conference: a young moderate willing to break with his leadership and his President on everything from gun control bills to repealing and replacing the Affordable Care Act. Representing the Philadelphia suburbs, Costello has remained left of many of his colleagues. However, in the Trump era, the congressman is reluctantly moving on, fed up with the polarization of not only Washington but also the country at large.

"I feel, increasingly, that if you're a member of Congress, they assume that you're not good," he told CNN in a recent interview. "They assume that you're not telling the truth. No matter what you do, someone's out there locked and loaded to say something disparaging, false, mean, in an attempt to have other people not like you."

But Costello -- unlike some of his older colleagues, who've decided to hang it up after decades-long careers or after being term-limited out of powerful committee chairmanships -- said t he Pennsylvania state Supreme Court's redrawing of his district so that it became more Democratic-leaning also was a major factor.

"I'm 41. I have an 8-month old. I have a 4-year old. I just lost half my congressional district. The state Supreme Court decided that they were going to try and take me out," Costello said.

Another factor? The reality that no matter what day it is in the Capitol, he typically is inundated with questions about Trump.

"No matter what I say or do, I feel all I do is answer questions about Donald Trump rather than health insurance or tax policy," Costello said. "I think it's a very challenging political environment, and when you add on top of that just the demands from a work-family balance, I just felt it was best for me to take stock in my life and have eight months to decide what I'm going to do next rather than, potentially, six weeks."

When he walks away, Costello said, he'll miss the camaraderie he felt with other members at times and the "pinch-me" moments, the experiences he couldn't have had if he weren't a member of the US House of Representatives.

"There is the occasional pinch-me moment. There's the occasional bang-my-head-against-the-wall moment, too, but there's the occasional pinch-me moment," he said. "That is something that I'll certainly miss."

Jeff Flake: Trump's foil stepping aside

JUST WATCHED Flake: the choice was embrace Trump, or retire Replay More Videos ... MUST WATCH Flake: the choice was embrace Trump, or retire 00:51

There are few Republicans in Washington who have been more uncompromisingly outspoken about Trump and shortcomings than Arizona's Sen. Jeff Flake.

From the earliest days of the presidential campaign, in which Trump suggested Mexican immigrants were rapists and Flake's Arizona GOP colleague, Sen. John McCain, didn't deserve the distinction of war hero because he'd been captured in Vietnam, Flake has been crystal clear that he thinks Trump's bombastic style is hurting the country and endangering America's reputation abroad.

It's Trump and what he represents about the far-right and nativist underpinnings of Flake's beloved Republican Party right now that drove the 55-year-old senator to decide that now was as good a time as any to retire from Congress.

"Our debt has increased massively; we're no longer the party of free trade. We've become a party of anger and resentment, and that is not the optimistic vision that I cut my teeth on as a Republican listening to Ronald Reagan or George H.W. Bush or others," Flake said.

After a single term in the Senate and more than a decade in the House of Representatives, Flake will leave at the end of the year just after his youngest son has graduated from high school and he and his wife, Cheryl Flake, are empty-nesters.

"My wife reminded me the other day that we had done this completely backward. I thought if I ever run for office, when I was younger, much younger, that it would be after the kids were grown," Flake said. "Instead we raised our kids during this period and now they're leaving the nest and I'm retiring."

Flake said the first thing he plans to do is take a big trip (maybe a couple), similar to the exit strategy of former President Barack Obama.

"I will go (to) an island or two. I've done that in the past, even when I've been in session, just to find a break and as a hobby," he said. "Go out and survived on an island with no food or water. That's my kind of idea of fun."

As he heads for the exit, he leaves some unfinished business. Flake -- who has been among the Republicans who worked relentlessly on immigration restructuring and enshrining the Deferred Action for Childhood Arrivals program into law -- said he's frustrated he hasn't been able to see the legislation through. He also lamented that he's leaving at a time when the Senate isn't doing as much legislatively as he wishes it was, in part because of arcane Senate rules and in part because, he said, leadership on both sides of the aisle has shielded members from difficult votes.

"We end up in this dance where we just don't vote very much. That needs to change. Take a vote. Just take a vote," he said. "I hear sometimes from my colleagues, 'We can't do that because the President doesn't support it.' Ultimately the President has to sign or reject what we pass, but to obsess over whether the President will agree with what we've put on his desk just means that we put very little on his desk."

Behind the scenes, Flake said, there are many more Republicans as concerned about the President and the direction of the party as he is. From public policy to the Russia investigation, Flake said, members in the GOP are more vocal behind the scenes than they are before the cameras.

There is one issue where members are especially worried: No one wants Trump to fire special counsel Robert Mueller.

"All of us are quietly, and sometimes publicly, warning, or begging, or admonishing the President not to go that route. I certainly hope he doesn't," Flake said.

Ileana Ros-Lehtinen: The first Latina in Congress retires

JUST WATCHED 'I'm not a Trumpian. I'm a Republican' Replay More Videos ... MUST WATCH 'I'm not a Trumpian. I'm a Republican' 01:47

In the span of a single hour, Rep. Ileana Ros-Lehtinen's office was never not doing something.

The Floridian sat down with CNN, met with a landscape architect and entertained representatives from an after-school program for underprivileged kids, asking questions and engaging in a manner that many members who've been around Capitol Hill for as long as she has lost long ago. Everyone was offered Cuban coffee, perhaps the secret to Ros-Lehtinen's boundless energy and enthusiasm for a job she's held for nearly three decades.

Unlike other members walking away this cycle, Ros-Lehtinen doesn't have a list of gripes about her job, the hours, the fundraising, the endless mix of meetings. She loved it all, she said.

Instead -- as the first Latina ever elected to Congress (a fact she herself didn't know until after she'd won her seat) -- Ros-Lehtinen offers a glimpse into just how much Congress has changed for women and minorities in a body where they are steadily gaining ground.

"When I first got to Washington, there were not even 30 women. Not even 30 women in the House. I don't know what the number is now. I've lost count, there's so many of us, thank goodness. Is it parity? Is it what it should be? No. But is it getting better? You bet," she said.

Still, Ros-Lehtinen argues there is more work to do on some committees, like House Foreign Affairs, where she served her entire congressional career and once chaired.

JUST WATCHED An essential part of the job in Congress Replay More Videos ... MUST WATCH An essential part of the job in Congress 00:32

"We need to do better in getting more women involved in those kinds of committees, so that when the audience is looking at us, they're thinking, 'Oh, yeah. This is representative,' " she said.

When she retires, Ros-Lehtinen said, she hopes to split her time between Washington and Florida. While Miami is her primary residence, she said she has a "dumpy" townhouse in DC in a great location and wants to "save it for a rainy day." She's not interested in shunning politics altogether. Even in the era of Trump, who she admits has made Washington more polarized, she believes Congress has always been a tough place.

"It seems a little more antagonistic, so that part has changed a little bit, but it's not like this was 'Leave it to Beaver,' 'Mayberry R.F.D.,' 'The Donna Reed Show,' " Ros-Lehtinen said. "It was never a Pollyanna, Mary Poppins place. It was always rough and tumble politics."

The congresswoman leaves her post with very few regrets, although she admits she would have liked to have helped establish an American Latino Museum as well as passed her bill that would allow Holocaust survivors to present a case in court, something they aren't able to do. She also once aspired to have a perfect voting record, but that was shattered on her first day in Congress in 1989, when she got an unexpected invitation to meet with the father of her congressional campaign manager, George H.W. Bush and Jeb Bush, respectively.

"George Herbert Walker Bush was the President and he had campaigned for me in Miami, so the first day, I get sworn in and his office calls me up, the White House, and ... says, 'The President would like to see you,' " she recalled. "I said, 'Well, we have votes.' So the very first day, I missed like seven votes, so so much for my perfect voting record, which is something I wanted to do."