The city of Richmond, Calif., is one step closer to moving forward with a plan to seize underwater mortgages by invoking eminent domain.

Earlier this week, federal district judge, Charles Breyer, dismissed a lawsuit that was filed by Wells Fargo that aimed to stop the plan from moving forward. Wells Fargo was serving as the trustee for dozens of investment trusts in the suit.

Under Richmond's controversial plan, it would be able to buy 624 mortgages at the present market value of the homes from investors. The city then plans to modify the loans so borrowers can better afford their payments.

Related: Best Places to Live: Most affordable towns

The lower mortgage payments would not only enable some residents to avoid foreclosure, but it would also help reduce neighborhood blight and boost home values, said Steven Gluckstern, chairman of Mortgage Resolution Partners (MRP), the private company helping to finance the program for the city.

However, mortgage lenders oppose the deal, saying it will have a negative impact on the market going forward. They claim banks will stop issuing mortgages in communities where eminent domain has been invoked. In addition, eminent domain could also raise the costs of borrowing for all homebuyers by making it harder to sell mortgage bonds to risk-averse investors, who will insist upon charging higher rates on mortgages.

What's next?

With the green light from the judge, the city of Richmond's next step is to try to partner with other cities in order to split the costs. There are half a dozen potential candidates, ranging from El Monte and Vallejo, Calif., to places like Seattle that are exploring the idea, said Gluckstern.

After that, Richmond's city council will have to authorize the mortgage seizures. That may prove tough. When the council voted to further explore the idea last week, it passed with only a four-to-three majority. California law requires a super majority -- five votes to two in this case -- for such resolutions. And even if it passes, a court would still have to give final approval.

Related: Foreclosure crisis is drawing to a close

Richmond has targeted mortgages with balances averaging $370,000 for seizure. But some of the homes are much pricier than that. A few were even bought for $1 million or more, said Mark Joffe, a consultant with PF2 Securities Evaluations, who got the data on the homes through a California public records request.

"If they are trying to help struggling homeowners in areas at risk of blight, these properties just don't seem appropriate," he said.

One house was in the ritzy Point Richmond section. It originally sold for $1.2 million during the boom and has a balance of $888,000. The city said it would buy the mortgage for $510,000, according to the data Joffe received. If the city's plan were to go through, it would then lower the mortgage balance by almost a quarter-million dollars.

Related: Surprising foreclosure hot spots

Gluckstern said the homes were chosen to fit certain criteria, including that the loan be privately held -- none were backed by government agencies like Fannie Mae, Freddie Mac or the Federal Housing Administration.

He said the city is only resorting to eminent domain because previous offers to buy mortgages directly from lenders at the discounted values were rejected.

"If we could do this without going through this contentious process, we would," he said.