Washington(CNN) The Supreme Court seemed splintered Tuesday on the fate of the Consumer Financial Protection Bureau, an independent agency created to combat unfair and deceptive practices against consumers in the wake of the 2008 financial crisis.

The arguments -- some of the most lively of the term -- revealed that several of the conservative justices, including President Donald Trump's two nominees, had constitutional concerns about the leadership structure of the agency where a single director serves and cannot be removed by the President except for "inefficiency, neglect of duty or malfeasance."

At one point, however, Chief Justice John Roberts -- whose vote could be key-- questioned whether the court could avoid constitutional questions and rule more narrowly. The conservatives seemed divided among themselves, thinking out loud at times on how the case should be decided. And although there were concerns about the fact that a single director could be too insulated from the Executive Branch, there did not seem to be an appetite for a ruling that would strike down the agency in its entirety. The government pushed the court to strike the so-called "removal " provision of the law that created the CFPB but leave the rest intact.

For their part, the liberal justices repeatedly suggested that the case should be dismissed because it was improperly before the court because the justices can only decide issues that present an ongoing controversy. They noted that as things stand, under the Trump administration, the current director of the agency believes that she already serves at the pleasure of the president. Justice Sonia Sotomayor suggested that the court should wait until there is an actual dispute between a president and the leader of the agency before striking down even a small provision of an act of Congress that established the CFPB.

Justice Ruth Bader Ginsburg also threw her support behind the agency, calling the restraints on the President "modest" and pointedly noting that Congress crafted a single director to have independence from the President in order to better protect consumers.

After arguments, it was unclear how the court would ultimately rule in the case that could also impact the fate of other agencies, as well as the Trump administration's determination to rein in the power on federal agencies.

Mick Mulvaney, who served as Trump's first acting director of the CFPB and now is acting White House chief of staff, sat in the audience.

Critics of the leadership structure -- including the Trump administration, current CFPB Director Kathleen Kraninger, as well as a law firm fighting a CFPB-led investigation -- argued that the director has too much power and is insulated from removal if the President has policy disagreements.

Such a structure, the argument goes, encroaches on the President's authority over the Executive Branch.

The dispute comes as some conservatives, including Trump's two Supreme Court nominees Brett Kavanaugh and Neil Gorsuch, seek to diminish the power of agencies, arguing that they are unaccountable to the public and disrupt the separation of powers.

Supporters of the CFPB, on the other hand, say the bureau needs independence and discretion to protect the consumer.

The challenge is brought by Seila Law, a law firm that helps individuals resolve their debts, which has a client currently involved in a CFPB investigation. Kannon K. Shanmugam, a lawyer representing the law firm, argued his client declines to turn over documents requested by the agency because the CFPB is unconstitutionally structured, wielding an outsized "enormous power over American businesses, American consumers and the overall US economy."

Shanmugam said that the court should strike down the leadership structure and send the issue back to Congress for a fix. Alternatively, he said the court should strike the section of the sprawling law that created the bureau.

Solicitor General Noel Francisco agreed that the court should hold that the single director structure is invalid and violates the separation of powers.

"This case concerns whether Congress may restrict the President's ability to remove the single principal officer of an agency exercising substantial executive power," Francisco argued in court briefs, concluding: "It may not."

The Trump administration suggested that the CFPB can continue, but the Court should sever the part of the law that says the President can only remove the director "for cause" and not "at will."

That legal position aligns with that taken by Kavanaugh when he was a lower court judge and issued an opinion in a separate case concerning the bureau. Kavanaugh said in that opinion that the "massive power" and "absence of Presidential supervision" threaten individual liberty and the constitutional system of checks and balances.

In court, Francisco suggested that the single director could be replaced with a multi-member structure, something that received criticism from Justice Elena Kagan, who suggested that a president would actually have a harder time controlling a multi-member leadership team than a single director.

And Ginsburg appeared to criticize the government for its "uncommon" decision not to defend an act of congress. "How often has the [Solicitor General] declined to defend legislation adopted by Congress," she queried.

"The general rule," Francisco shot back, "is that we defend the acts of Congress unless it infringes upon the President's own executive power."

"Here we believe that we have a statute that infringes upon the President's own executive power," he said.

Roberts suggested that instead of issuing a ruling steeped in the separation of powers, the court could simply interpret the removal provision more broadly, a move that would stop the court from taking the step of striking a provision passed by Congress, but could limit the independence of the director.

That tack seemed to frustrate Gorsuch, who referred it to it as watering down the statute instead of something more definitive.

Because no party supported the current structure of the agency, the Court announced it would appoint one of the best appellate lawyers in the country -- former Solicitor General Paul Clement -- to do so.

As a threshold matter, Clement argued that the Court should sidestep the weighty issues of the case and take available offramps. One avenue concerned the fact that Kraninger, the current director of the CFPB, does not dispute that she should be removed at will if the President disagrees with her decisions.

"This case presents a remarkably weak case for invalidating an Act of Congress," Clement said, suggesting the court should wait to decide the issue when there is a director who is at odds with the President.

If the court were to reach the merits, Clement advised the justices that Congress has "repeatedly hewed to the view" that the discharge of certain specialized roles are best discharged by someone who "does not hold his office at the pleasure of the president."

When Warren initially contemplated the makeup of the CFPB, she envisioned a multi-member director format, similar to other federal agencies — a point Shanmugam brought up in court. But asked about the Supreme Court dispute while on the campaign trail last week, she said that when she was working to build the agency, "We did everything possible" to insulate it from the "whims of an individual president, because we wanted that agency to work on behalf of the people."

Under the Trump administration, she said, "the agency hasn't been as aggressive as I think it should be" but it is still out there, "it's still a cop on the beat."

This story has been updated to reflect Tuesday's arguments.