When it comes to residential mortgages, big banks are waving the white flag.

Banks originated 74% of all mortgages in 2007, but their share fell to 52% in 2014, the most recent data available from the Mortgage Bankers Association. And it could go even lower.

But even at these levels, the big bank backtrack is reshaping a lending landscape that’s already undergone seismic shifts since the housing bubble burst.

While there’s widespread agreement that banks should have been reined in — and perhaps punished — after playing a major role in the housing bubble that helped tank the economy, the past few years have been tough for banks’ mortgage businesses.

They now face a regulatory environment so strict that many are afraid to lend, even to customers with the most pristine credit. They’re still paying up for misdeeds done during the bubble. There’s essentially no private bond market to whom to sell mortgages.

And fighting those battles on behalf of their least-profitable divisions means residential lending just isn’t worth it for many banks.

“We can’t make money in the business,” BankUnited CEO John Kanas said when he announced a mortgage retreat on a January earnings call. “We realized that this was the lowest-margin, most volatile business we had and we decided that we should exit.”

Of the top 10 originators in 2015, banks lent 28.6% of all mortgages, according to data from Inside Mortgage Finance. That’s about half their share in 2012, when banks among the top 10 originators accounted for 54.4% of all mortgages.

Lender 2015 Volume, billions Market share Wells Fargo $209.01 12% Chase $115.22 6.6% Quicken Loans $78.50 4.5% Bank of America $56.92 3.3% U.S. Bank $52.81 3% PennyMac $48.39 2.8% PHH Mortgage $37.57 2.2% Freedom Mortgage $36.80 2.1% Citi $34.39 2.0% Flagstar Bank $29.37 1.7%

For many analysts, that step is only natural.

“The fact is that the cost of capital and compliance has convinced many bankers that making home loans to American families is not worth the risk,” said Chris Whalen, a long-time bank analyst now with Kroll Bond Rating Agency, in a speech early in February.

Whalen expects the four largest commercial banks will “down size or exit entirely from the business of originating and servicing residential mortgages.”

The number one lender, Wells Fargo, WFC, -0.31% originated $47 billion in mortgages in the fourth quarter of 2015, compared to $125 billion in the last three months of 2012.

J.P. Morgan Chase JPM, -0.77% lent $22.5 billion in mortgages in the fourth quarter, down from $51.2 billion in the same quarter three years ago.

Other experts aren’t sure the banks will exit “entirely.”

Ted Tozer is the president of Ginnie Mae, the mortgage giant that backs loans sold by the Federal Housing Agency and the Veterans Administration.

Tozer thinks banks have already whittled down to about the level of lending they want to keep up. And he believes many big banks wouldn’t turn down the opportunity to offer a mortgage to an existing customer — to be “full-service,” as he says.

But Tozer also sees the mortgage market going “back to the future.” Big banks have dominated mortgage lending only since the 1990s, when economies of scale for technological infrastructure made the “financial supermarket” model attractive, forcing smaller players to take a back seat.

Community banks and credit unions are going to step into the void, Tozer told MarketWatch, although they won’t be big players because post-crisis regulations limit how much mortgage debt they can hold. “Right now they should be flourishing, but they won’t be able to live up to their full potential” because of Basel requirements, Tozer said.

The biggest players will instead be online lenders, sometimes called “nonbanks.”

In 2015, four of the top ten originators were such entities, according to data from Inside Mortgage Finance – Quicken Loans, PennyMac Financial, PHH Mortgage, and Freedom Mortgage. Those institutions, also known as “independent mortgage bankers,” made up 43% of all originations in 2014, according to the Mortgage Bankers Association, a share that’s stayed steady or grown every year since 2007, when it stood at 23%.

While Quicken drew some fire for a Super Bowl commercial that seemed to suggest getting a mortgage should be as easy as it was during the housing bubble, nonbanks must operate within the same rigorous underwriting framework that banks do post-crisis.

And many of them are flourishing because they were created after the crisis to clean up delinquent loans, meaning they know how to service the more challenging loans.

Ellen Seidman, senior fellow at the Urban Institute, sees a future mortgage market that’s somewhat bifurcated between small, local institutions like community banks and credit unions, and online lenders, whom borrowers might never meet in person.

Seidman thinks that bifurcation will take place along age lines, with nonbanks picking up “a larger and larger share of tech-savvy millennial population. They may get steered wrong, it certainly has happened before, but there are a lot of people trying to do it right,” she said.

A more serious question, Seidman said, is whether minority and immigrant communities will be well-served in the future. That’s where other questions, like whether lenders are responsible for the bad loans they make, come into play.

“It is worth worrying about how are we going to not have a repeat of some of the bad stuff done during the bubble,” Seidman said. “There are reasons to believe it will be harder to make those mistakes again, but if we get into a mode of trusted people being able to originate mortgages without having responsibility for them, for their performance, it’s not clear.”