The structure of the Consumer Financial Protection Bureau is constitutional, an appeals court ruled Wednesday in a blow to President Donald Trump's efforts to ease regulations on the financial system.

The U.S. Court of Appeals for the District of Columbia Circuit made the ruling in a battle over whether the president could remove the director at will. The court in October had upheld a challenge to the structure but agreed to rehear the case.

Republicans had challenged the CFPB structure on grounds that the director's position was unaccountable to the executive branch.

PHH Corp. had filed the initial suit in a fight over a $109 million fine the bureau leveled against the company for allegedly taking kickbacks for sending customers to mortgage insurers. The CFPB added $103 million onto an initial $6 million fine, something it lacked authority to do, PFF charged.

The court did say the CFPB made errors in assessing the added penalty. The case could now be headed to the Supreme Court.

Trump has been a critic of the bureau, whose initial director, Richard Cordray, stepped down in late 2017. Office of Management and Budget Director Mick Mulvaney has led the bureau since, though there had even been a dispute over whether he or Leandra English, Cordray's chief of staff, is the rightful leader. Wednesday's decision does not address that issue.

The appeals court's ruling likely doesn't end the long-running dispute over the bureau.

Richard Hunt, CEO of the Consumer Bankers Association, said "we applaud" the decision but the group sees further adjustments as warranted.

"Congress should create a bipartisan commission at the CFPB, in place of a sole director, to uphold the Bureau's mission of consumer protection and would establish transparency, diversity of thought, additional industry insight and rule makings beneficial to consumers, the industry and the economy," Hunt said in a statement.

The president has said the bureau has overreached in its enforcement of financial system regulations. Trump has sought a less restrictive environment on a number of fronts but particularly banking, which he sees as key to generating more economic development.

In making its ruling, the court rejected the argument that the president could remove the CFPB director without cause. The ruling stated that a provision of the Dodd-Frank reforms, passed after the financial crisis of 2008, "shielding the Director of the CFPB from removal without cause is consistent" with Article II of the Constitution.

Dodd-Frank states that the director can be removed only for "inefficiency, neglect of duty, or malfeasance in office."

CNBC has reached out to the White House for comment.