Tom Pennington / Getty Images Federal Reserve Bank of Dallas President Richard W. Fisher.

Since 2008, there have been plenty of calls to forcibly dismantle the “Too Big To Fail” (TBTF) Banks, but few of those calls have come from those in positions of real power. But the Dallas Federal Reserve Bank President, Richard Fisher, has been consistent and vocal in his belief that our economy will not reach its full potential until those bloated financial institutions are cut down to size.

(MORE: Should We Find Comfort in the Fed’s Stress Test Results?)

Yesterday, the Dallas Fed Bank released it’s 2011 annual report entitled “Choosing the Road to Prosperity: Why We Must End Too Big to Fail – Now.” The report was written by Harvey Rosenblum, the head of the Dallas Fed’s research department, and was introduced and endorsed by Fisher himself. It presents one of the most cogent arguments for the need to forcibly downsize TBTF banks, and it’s certainly one of the few calls for this approach coming from such a powerful federal regulator. In the introduction to the report, Fisher writes:

“The TBTF institutions that amplified and prolonged the recent financial crisis remain a hindrance to full economic recovery and to the very ideal of American capitalism. In my view, downsizing the behemoths over time into institutions that can be prudently managed and regulated across borders is the only appropriate policy response.”

Sounds nice, but how exactly can we force the banks to be smaller? Fisher and his head of research don’t really provide much of an answer, and readily admit that legislating the size of banks would be a difficult. Writes Rosenbaum:

“The prospect raises a range of thorny issues about how to go about slimming down the big banks. Second, the level of concentration considered safe will be difficult to determine. Is it rolling things back to 1990? Or 1970? Third, the political economy of TBTF suggests that the big financial institutions will dig in to contest any breakups.”

It’s one thing to say we shouldn’t have TBTF institutions; it’s another to figure out how to get there. To be fair, it’s not really the place of Fed officials to be too explicit with policy recommendations, as it undermines the impartiality many see as necessary for an effective central bank. But without a clear roadmap to downsizing the banks, calls for eliminating TBTF will amount to little more than howling at the moon.

(MORE: TIME’s Interview With Warren Buffett)

There is one possible pathway to ending TBTF that the report hints at, however. The Dodd-Frank law allows for the Financial Stability Oversight Council to designate certain institutions as “systemically important.” Critics deride this section of the law as one that institutionalizes Too Big Too Fail. If the government is officially designating which institutions are systemically important, they argue, that institution and the market will assume that the government will bail it out, leading to the same sorts of distortions that, in part, led to the financial crisis in the first place.

But at least in theory, if FSOC impliments the law optimally, being a systemically important financial institution could actually be a bad thing. Dodd-Frank gives regulators the ability to require higher capital levels for these institutions. If these requirements are stringent enough, banks will do whatever they can to make sure that they’re not systemically important – i.e. get smaller. While this would require the perfect balance of regulations, it may be the best path forward to eliminating To Big To Fail. With Congress gridlocked as it is, Dodd-Frank may be the only tool we have for preventing these behemoths from stifling the recovery and distorting our capitalist system.