To listen to some commentators is to believe that markets are the solution to all of our problems. Health care not working? Bring in the private sector. Need to rebuild a war-torn country? Send in the private contractors. Emergency relief after earthquakes, hurricanes, and tornadoes? Wal-Mart with a contract is the answer.

Whatever the problem, the private sector - markets and their magic - beats government every time. Or so we are told. But this is misplaced faith in markets. There is nothing special about markets per se - they can perform very badly in some circumstances. It is competitive markets that are magic, though even then we have to remember that markets have no concern whatsoever with equity, only efficiency, and sometimes equity can be an overriding concern.

In order to work their magical efficiency, markets need very special conditions to be present. There must be full information available to all participants. Product quality, locations and prices of alternative suppliers, every relevant piece of information must be known. Not quite sure if the wine is good or not? That's an information problem. Not sure if the used car has problems? Don't know where any gas stations are except the ones beside the freeway in a strange town? No way to monitor the quality of the building built in Iraq with U.S. aid? No way to be sure if consultants are worth the amount they are being paid? Information problems are common and they can cause substantial departures from the perfectly competitive, ideal outcome.

There also must be numerous buyers and sellers, enough so that no single buyer or seller's decisions can affect the market price. For example, if a firm can affect the market price by threatening to limit supply, the market does not satisfy this condition. If, as some claim, CEOs are in such short supply that they can individually negotiate their compensation, then the market is not producing an efficient outcome. Whenever there are a small number of participants on either side of the market - suppliers or demanders - this is potentially problematic.

In order for markets to work their magic, the product must be homogeneous. That is, the product or input to production sold by all firms in the market must be perfectly substitutable so that as far as the buyer is concerned, one is as good as the other. If some buyers favor one brand over another, if CEOs are perceived to have different and unique talents, if government favors one contractor over another due to political contributions, this condition does not hold. In many cases the variety may be worth the inefficiency, not many of us would want just one style and color of shirt to be available in stores, but the inefficiency is there nonetheless.

In order for markets to work their magic there must be free entry and exit. Most people understand free entry, but free exit is sometimes less evident, so let me try to give an example. Starting a blog on Blogger or TypePad is easy. Entry is a snap and you can be up and running in no time at all. It's easy to join the competition and start supplying posts. But suppose that later you decide you want to switch to, say, TypePad from Blogger (or the other way around). That is not so easy. There is no way, at least no simple and convenient way, to export all of your old posts from Blogger and import them into TypePad, a significant barrier to exit if a large number of posts must be moved. Whenever barriers exist in markets that prevent free movement into and out of the marketplace or between firms within a market (on either side - there are sometimes barriers to purchasing as well), markets will underperform.

The list goes on and on. In order for markets to work their magic, there can be no externalities, no public goods, no false market signals, no moral hazard, no principle agent problems, and, importantly, property rights must be well-defined (and I probably missed a few). In general, the incentives that the market provides must be consistent with perfect competition, or nearly so in practical applications. When the incentives present in the marketplace are inconsistent with a competitive outcome, there is no reason to expect the private sector to be efficient.

Markets don't work just because we get out of the way. When government contracts are moved to the private sector without ensuring the proper incentives are in place, there will be problems - waste, inefficiency, higher prices than needed, etc. There is nothing special about markets that guarantees that managers or owners of companies will have an incentive to use public funds in a way that maximizes the public rather than their own personal interests. It is only when market incentives direct choices to coincide with the public interest that the two sets of interests are aligned.

If there is no competition, or insufficient competition in the provision of government services by private sector firms, there is no reason to expect the market to deliver an efficient outcome, an outcome free of waste and inefficiency. Why would we think that giving a private sector firm a monopoly in the provision of a public service would yield an efficient outcome? If the projects are of sufficient scale, or require specialized knowledge so that only one or a few private sector firms are large enough or specialized enough to do the job, why would we expect an ideal outcome just because the private sector is involved? If cronyism limits the participants in the marketplace, why would we expect an outcome that maximizes the public interest?

There is nothing inherent in markets that guarantees a desirable outcome. A market can be a monopoly, a market can be perfectly competitive, a market can be lots of things. Markets with bad incentives produce bad outcomes, markets with good incentives do better.

I believe in markets as much as anyone. But the expression free markets is often misinterpreted to mean that unregulated markets are all that is required for markets to work their wonders and achieve efficient outcomes. But unregulated is not enough, there are many, many other conditions that must be present. Deregulation or privatization may even move the outcome further from the ideal competitive benchmark rather than closer to it, it depends upon the characteristics of the market in question.

For government goods and services, when incentives consistent with a competitive outcome are present, we should get government out of the way and privatize, and there are lots of circumstances where this will be appropriate. There is no reason at all for the government to produce its own pencils and pens, buying them from the private sector is more efficient so long as the bids are competitive.

When competitive conditions are not met but can be regulated, the regulations should be put in place and the private sector left to do its thing (e.g. mandating that sellers disclose problems with a house to prevent asymmetric information or mandating that government funded projects be subject to competitive bidding and monitoring to ensure contract terms are met). There's no reason for government to do anything except ensure that the incentives to motivate competitive behavior are in place and enforced.

But rampant privatization based upon some misguided notion that markets are always best, privatization that does not proceed by first ensuring that market incentives are consistent with the public interest, doesn't do us any good. There are lots of free market advocates out there and I am with them so long as we understand that free does not mean the absence of government intervention, regulation, or oversight, even libertarians agree that governments must intervene to ensure basics like private property rights. Free means that the conditions for perfect competition are approximated as much as possible and sometimes that means the presence - rather than the absence - of government is required.