In January, Richard W. Fisher, president of the Federal Reserve Bank of Dallas, said that Wall Street banks, "… as a result of their privileged status, exact an unfair tax upon the American people."

If a president of a regional Federal Reserve bank (hardly a socialist), is telling us that Wall Street is not only privileged, but using its privileges to "tax" the hell out of us, it might be wise to take a closer look. While he's not literally talking about a tax, he's talking about a kind of extortion — a virtual tax that ends up in the pockets of our Wall Street barons.

While our political elites warn us about becoming the next Greece (do we get the tasty retsina, good weather and nice beaches too?), the real "tax and spend" game is taking place on Wall Street, hidden from view. In fact, Wall Street is essentially exerting a hidden tax on us day in and day out — but we don't even notice or get anything back in return.

Before describing how Wall Street does it, let's try to wrap our minds around how big our biggest banks really are. In 1970, the top five U.S. banks owned 17 percent of all U.S. banking assets. By 2010, well after the crash, the top five banks owned 52 percent of all our banking assets.

Who are these giant banks today? The usual suspects — JP Morgan Chase, Bank of America, Goldman Sachs, Citigroup and Morgan Stanley. These five alone have assets that account for more than one quarter of the entire U.S. economy. They form an oligopoly, defined by Investepedia.com as "a situation in which a particular market is controlled by a small group of firms. An oligopoly is much like a monopoly, in which only one company exerts control over most of a market. In an oligopoly, there are at least two firms controlling the market."

As a result, a very small group of banks can exert power over markets to boost profits and gouge consumers — that would be us. The extra we pay to them is precisely like a tax, except we don't know we're paying it. This puts an entirely new spin on "no taxation without representation."

Here are four Wall Street ploys that create a hidden and unfair tax on all Americans.

1. Too-big-to-fail banks jack up every mortgage rate in the country.

The largest U.S. banks use their oligopoly power to siphon money from the mortgage markets. This means they can charge consumers higher interest rates for loans and credit card debt, and they can keep interest rates on home mortgages higher than they should be. For example, the five largest banks are the primary buyers of loans originated by community banks and mortgage companies. It's still enormously prosperous for giant banks, but the smaller community banks are squeezed. As Jeff Horowitz writes in the American Banker, "The country's biggest banks are again making handsome profits from buying and servicing home mortgages. The smaller institutions that produce the loans have not been so fortunate."

Neither has the consumer. Although mortgage rates are low because of the financial crash, they would be even lower if the big banks did not extract extra oligopolistic profits. It's just a fact of economics that when competition among banks declines, the cost of mortgages to consumers goes up. How much? This amounts to a hidden tax of $1,000 per year for anyone taking out a 30-year mortgage. But this hidden tax is so subtle that the average consumer has no idea that she is being fleeced. As the New York Times, reports, "Banks are making unusually large gains on mortgages because they are taking profits far higher than the historical norm, analysts say. That 3.55 percent rate for a 30-year mortgage could be closer to 3.05 percent if banks were satisfied with the profit margins of just a few years ago. The lower rate would save a borrower about $30,000 in interest payments over the life of a $300,000 mortgage."

2. The biggest banks and their hedge fund partners engage in high-speed trading that rips off everyone who has money in pension and mutual funds.

In doing research for my latest book, How to Make a Million Dollars an Hour: Why Hedge Funds Get Away with Siphoning Off America's Wealth I came across the incredible phenomena of high-speed computer trading. Wall Street makes somewhere between $5 billion and $20 billion a year using ultra-fast and expensive computers that can execute millions of trades per second.

Why make so many trades so quickly? You may find this almost impossible to believe, but by being able to trade in nanoseconds, the high-speed computers can "see" when you're about to buy a stock, get there before you and then resell it back to you for a few pennies more. Between the time you press the "buy" button to the time your trade goes through, the price is bumped up a few pennies by these automated high-speed trading systems. The same goes for your mutual and pension funds. When they buy or sell stocks in your behalf, the high-speed computers are waiting to pounce in order to extract a few pennies from each and every trade, just like a sales tax — except that it goes into Wall Street's ever deepening pockets.

3. Insider trading is yet another hidden tax.

The U.S Attorney's office in Manhattan has nailed over 70 hedge fund employees on charges of insider trading. Since these cases are extremely difficult to prosecute, it's reasonable to assume that we're seeing a very small tip of the corruption iceberg. No doubt there are hundreds of other culprits, if not thousands, throughout Wall Street who are illegally profiting from insider trading.

But who's the victim? You are. Again, it's likely to come out of your pension and mutual funds. When a Wall Street money manager makes a lucrative trade based on illegal insider information, those on the other side of the trade are worse off. Either you don't get the upside or you are stuck with the downside. When the definitive history of this period is finally written, it may show that most, if not all, of the super-profits enjoyed by hedge funds and the proprietary trading desks of too-big-to-fail banks over the past decade came from illegal financial machinations.

4. The biggest tax of all? As Wall Street grows larger, economic growth is harmed, thereby killing jobs, decreasing tax revenues and provoking debt crises.

Andrew Haldane, a key regulator for the Bank of England, reports on an amazing study which shows that as banks grow larger and more concentrated, economic development is harmed. "There is a threshold at which private credit-to-GDP may begin to have a negative impact on GDP growth," writes Haldane. "That threshold is found to lie at a private credit-to-GDP ratio of around 80-100%."

Unfortunately, our private credit-to-GDP ratio is about 200 percent, meaning the private debt created by banks is twice the size of our economy and doing more harm than good. Let me put this crudely. When the financial sector grows too large, it sucks the economic life out of the economy. Its hidden taxes siphon away investment money from other sectors that could use it much better. It milks consumers and lowers effective demand. And it uses its too-big-to-fail status to take excessive risks to boost profits, knowing full well that any major downside — the inevitable failures — will be covered by the taxpayer.

Wall Street's Hidden Taxation has Too Much Representation

These hidden taxes exist because both political parties are snuggled under Wall Street's blanket of greed. Instead of blaming Social Security, Medicare or Medicaid, we need Washington to finally admit that the financial problem is Wall Street. These banks and hedge funds are much too large. The hidden taxes are unfair and undemocratic. It's time to reclaim them.

The solutions are simple.

Break up the big banks: Even the president of the Dallas Fed is calling for the dismemberment of too-big-to-fail banks. This goes far beyond the so-called Volker rule or the return of Glass-Steagall, which separates consumer banking (our insured deposits) from investment banking (their casinos). These giant financial entities have to be broken up into bite-size chunks so that no one firm can blackmail the economy into a bailout. Capitalism has a lot of problems. But if you take away the fear of bankruptcy, you exacerbate its worst tendencies. Give the big banks the power to rip us off and they will, no matter what their eloquent CEOs say to Congress about their ability to manage risk.

Pass a financial transaction tax (Robin Hood tax): Big banks and their hedge fund cousins are using high-speed trading to extract a hidden tax. Let's make that tax overt and have it go to the commonwealth instead of into Wall Street's pockets. It will reduce, if not eliminate, the high-speed predator trading and help provide the funding we need to build a more just and equitable society. (The New York Times reports on 1/22/13 that several European countries will soon implement such a tax.)

Defeat each and every politician who takes money from Wall Street: I know, I know. If we did that, Congress would nearly be empty. But we need to break the incestuous links between Washington and Wall Street. The Democrats, in particular, must be forced to realize that they can't be the party of both the predators and the prey. We need to remind them that when they say the word "debt" it all starts on Wall Street. There would be no talk of a so-called debt crisis were it not for the Wall Street crash born and bred within too-big-to-fail banks.

A true political fighter for Main Street should be furious that these banks and their hedge fund cousins took record profits when they pumped up the housing bubble, then profited again by helping it burst, and then profited yet again while gorging on bailouts. Meanwhile, 9 million jobs were destroyed. Instead of setting their sights on Social Security and Medicare, politicians should go after the biggest entitlement in history — too-big-to-fail banks.