President Obama proposed a tax on the country's largest banks to help recover the money lost under the Troubled Assets Relief Programme (Tarp). This tax is a positive step. However, it will not come close to recovering the losses incurred in the bailouts and it will do almost nothing to change the way that the banks do business. For this we will need a larger financial speculation tax.

First, it is necessary to be clear on the extent of the losses incurred in the bailouts of the financial system. The losses in the Tarp are currently pegged at close to $120bn, mostly due to the bailout of AIG, the giant US insurance company. This money was virtually a direct handout to several large banks, as the government's money allowed AIG to make payments to Goldman Sachs and other large banks that would not have been possible if it had fallen into bankruptcy.

But these losses are far from the complete picture with the Tarp. On the night before Christmas, the Treasury department lifted the $200bn cap on the amount that both the mortgage agencies Fannie Mae and Freddie Mac can draw on the Treasury. They both now have unlimited lines of credit.

No one knows how much their bailouts will eventually cost taxpayers, but it is almost certain that their losses are not entirely attributable to the portfolio that the mortgage giants held on 7 September 2008 when they were put into government conservatorship. Many of the losses incurred by Fannie and Freddie are almost certainly due to losses on mortgages they purchased from banks after they went into conservatorship. In other words, Fannie and Freddie were paying too much for the mortgages they purchased from the banks. This is exactly what the Tarp was originally supposed to do.

In effect, the treasury department has run a version of Tarp through Fannie and Freddie. If we want to calculate the money taxpayers lost through from the Tarp programme we should certainly include the money lost bailing out these mortgage giants, which can now exceed $400bn if events turn out badly. This means that if the point is to recover the money lost in the Tarp, the bank tax is likely to fall short by a large margin.

The other key consideration in making the banks pay should be to structure a tax that changes the way the banks do business. This money lost in the Tarp programme is just a small fraction of what the banks' greed cost the country. We will likely lose more than $4tn in output in this downturn, more than 40 times the projected revenue from the tax over the next decade.

The $9bn that is projected to be collected each year is equal to about 5% of their annual profits and bonuses. It is unlikely to have any noticeable impact on the way they do business. In other words, we can still expect them to be pursuing short-term profits and giving little consideration to long-term investments.

A tax on financial speculation more generally, which will also apply to hedge funds and other financial institutions, would be a far more effective mechanism in changing behaviour. It could also raise very substantial revenue. In the UK, a tax of 0.25% on the purchase and sale of shares of stock raises the equivalent of $30bn annually in the US relative to the size of its economy. A broadly based transactions tax – that would apply not only to stock, but also to options, futures, credit default swaps and other financial instruments – could raise more than $150bn a year in the US.

Such a tax would also make the financial sector more efficient by reducing the volume of short-term trading that serves no productive purpose. The share of the private sector that is devoted to investment banking and commodities trading has nearly quadrupled in the last three decades.

By reducing the volume of trading this tax would make the financial sector more efficient, freeing up resources for productive uses. This would be comparable to improving the trucking sector by reducing the number of trucks and drivers it takes to deliver goods to wholesalers and retailers. Industries are supposed to become more efficient as the economy develops. It is only finance that is becoming less efficient due to its ever-growing complexity.

In short, a tax on financial speculation is a win for just about everyone but the speculators. President Obama's bank tax is a good start but we have to go much further.