Or: Why the Heritage Freedom Index is a Damned Statistical Lie

This guest post was contributed by StatsGuy, a frequent commenter and occasional guest on this blog. It shows how quickly the headline interpretation of statistical measures breaks down once you start peeking under the covers.



Recently, a controversy raged in the blogosphere about whether neo-liberalism has been a bane or a boon for the world economy. The argument is rather coarse, in that it fails to distinguish between the various elements of neo-liberalism, or moderate deregulation vs. extreme deregulation. But if we take the argument at face value, one of the major claims of neoliberals is that countries in the world which are more neoliberal are more successful (because they are more neoliberal). I disagree.

My disagreement is not with the raw correlation between the Heritage Index and Per Capita GDP. A number is a number. My disagreement is with the composition of the index itself, and interpreting this correlation as causation between neo-liberalism and ‘good things.’

My primary contention below is that many of these measures used in the composite Heritage Index have nothing to do with less government, and a lot more to do with good government. It is these measures of good government that correlate to economic growth and drive the overall correlation between the “Freedom Index” and positive outcomes. Secondarily, I will argue that many of the other items in the index (like investment freedom) are not causes of growth, but rather outcomes of growth.

The Heritage Index weighs ten items equally, and these items are derived using very different mechanisms (many subjective):

Business Freedom

Trade Freedom

Fiscal Freedom

Government Freedom

Monetary Freedom

Investment Freedom

Financial Freedom

Property Freedom

Freedom from Corruption

Labor Freedom

Here are the top twenty countries on the index, and their overall scores.

Hong Kong 89.7 Singapore 86.1 Australia 82.6 New Zealand 82.1 Ireland 81.3 Switzerland 81.1 Canada 80.4 United States 78 Denmark 77.9 Chile 77.2 United Kingdom 76.5 Bahrain 76.3 Mauritius 76.3 Luxembourg 75.4 The Netherlands 75 Estonia 74.7 Finland 73.8 Iceland 73.7 Japan 72.9 Macau 72.5

It strikes many that some countries we do not normally consider very libertarian rank quite high – Singapore (with the government actively manipulating the property market through its monopoly on land sales, or deploying investments through its massive sovereign wealth fund, etc.), Canada (with its state run health system), and others.

Likewise, on the negative side of the equation, we can observe several countries that score in the upper third on the Heritage Index that strike us as weak states. It turns out that Jamaica scores a rank of #58, and Colombia a #57. This is well above a lot of other countries like Argentina (#135), and even slightly above relatively successful nations like France (#63), Poland (#70), and Italy (#73). These types of scores tend to raise questions.

If we look under the hood, it turns out that Jamaica would score even better on the Heritage Index if it weren’t for two components on which it scores miserably. Here are Jamaica’s scores:

Overall Score 65.5 Business Freedom 87 Trade Freedom 72.2 Fiscal Freedom 74.8 Government Spending 61.8 Monetary Freedom 68.4 Investment Freedom 85 Financial Freedom 60 Property Rights 45 Freedom From Corruption 31 Labor Freedom 70

Those two scores are massively dragging down the average. This immediately makes one wonder how the various components relate to each other. Let’s take a look at the simple correlations:

Overall Score Business Freedom Trade Freedom Fiscal Freedom Government Spending Monetary Freedom Investment Freedom Financial Freedom Property Rights Freedom From Corruption Labor Freedom Overall Score 1.00 0.80 0.66 0.04 -0.12 0.60 0.83 0.85 0.83 0.79 0.45 Business Freedom 0.80 1.00 0.46 -0.06 -0.27 0.43 0.57 0.61 0.72 0.69 0.44 Trade Freedom 0.66 0.46 1.00 0.00 -0.22 0.36 0.61 0.58 0.49 0.51 0.15 Fiscal Freedom 0.04 -0.06 0.00 1.00 0.43 -0.30 -0.16 -0.09 -0.28 -0.32 0.13 Government Spending -0.12 -0.27 -0.22 0.43 1.00 -0.12 -0.27 -0.23 -0.38 -0.40 -0.13 Monetary Freedom 0.60 0.43 0.36 -0.30 -0.12 1.00 0.51 0.56 0.59 0.56 0.13 Investment Freedom 0.83 0.57 0.61 -0.16 -0.27 0.51 1.00 0.80 0.70 0.68 0.13 Financial Freedom 0.85 0.61 0.58 -0.09 -0.23 0.56 0.80 1.00 0.72 0.67 0.24 Property Rights 0.83 0.72 0.49 -0.28 -0.38 0.59 0.70 0.72 1.00 0.94 0.29 Freedom From Corruption 0.79 0.69 0.51 -0.32 -0.40 0.56 0.68 0.67 0.94 1.00 0.29 Labor Freedom 0.45 0.44 0.15 0.13 -0.13 0.13 0.13 0.24 0.29 0.29 1.00

Wow, there are some peculiar things about this table! Note, for instance, that Government Spending is negatively correlated with just about everything (except Fiscal Freedom). Note, also, that Property Rights and Freedom From Corruption have the highest bivariate correlation on the table (0.94). What if we were to try to visualize the relationships between the Heritage Index components, for example using a tool like MDS? We might get this (depending how you run it):

At first glance, it looks there are three groups of items: one focusing on property rights, corruption, and financial “freedom”; one focusing on structural factors in the economy (trade, business, labor, monetary); and one focusing on government spending (with fiscal freedom somewhere between government spending and the structural factors).

Let’s dig even deeper. If we open up the criteria for Financial and Investment Freedom, we find that both of these are subjective measures – using a point system in which Heritage deducts points depending on government interference with investments and finance. Subjective measures are problematic, because the individuals assigning points probably have a high bias to (unintentionally) favor countries that are successful overall. (That’s why drug trials require double blind tests.) But let’s give them the benefit of the doubt.

For Investment Freedom, Heritage penalizes points for several reasons, but half of the measure relates to legal recourse against appropriation of investor funds, bureaucratic controls over capital movement, and lack of transparency in investment law. . . . These sure sound a lot like Property Rights and Freedom From Corruption. For Financial Freedom, Heritage uses a 0-100 scale that is subjectively assigned based on the reviewer’s perceived influence of the government on finance. On this scale, the United Kingdom, which unleashed massive Quantitative Easing (not that I’m critical) and has tight links between large banks and Parliament scored an 80, near the top of the chart.

What about the variables in the lower left quadrant? Well, some of them we can probably ignore, like Monetary Freedom, which basically measures inflation and price controls. If we look at the standard deviation of these 10 measures across all countries, we get an idea of the typical influence of each variable on the final index score.

Measure Standard Deviation Business Freedom 18.0 Trade Freedom 12.1 Fiscal Freedom 12.7 Government Spending 20.4 Monetary Freedom 6.9 Investment Freedom 22.0 Financial Freedom 19.0 Property Rights 24.0 Freedom From Corruption 21.0 Labor Freedom 17.2

Monetary Freedom is flat, meaning that on average it accounts for little variation in the index score among countries. Property Rights and Freedom from Corruption typically account for three times as much variation in the final index. Even Trade Freedom and Fiscal Freedom are relatively flat compared to Property Rights and Freedom From Corruption. As one can see, the final index is (empirically) heavily weighted toward what I will call the ‘Good Government’ variables, rather than the ‘Less Government’ variables.

But let’s go just one step further. Let’s pull a measure of quality of life from the World Bank that uses something other than GDP. We’ll use Life Expectancy. While this is surely influenced by genetics, one must imagine it’s a pretty decent measure of quality of life as well. How does this measure correlate with the ten ‘Freedom’ measures above?

Measure Correlation With Life Expectancy Overall Score 0.59 Business Freedom 0.60 Trade Freedom 0.53 Fiscal Freedom -0.01 Government Spending -0.27 Monetary Freedom 0.30 Investment Freedom 0.46 Financial Freedom 0.47 Property Rights 0.61 Freedom From Corruption 0.64 Labor Freedom 0.17

Wow . . . Fiscal Freedom and lower Government Spending negatively correlate with life expectancy, even without controlling for other variables! A purist might observe that government spending could be an outcome, rather than a cause (i.e., government spending is a luxury good that wealthy societies engage in by mistake, but it doesn’t improve mortality rates). That sounds fishy, but even so, it still undercuts the neoliberal argument (less government = always better). Likewise, we observe very low correlations with Labor Freedom and Monetary Freedom, and one could argue that most of that correlation is spurious (that is, working through other factors).

Where are the highest correlations? Property Rights and Freedom from Corruption. . . . Again, our best measures of ‘Good Government’ in this index.

The second strongest pair of measures – Business Freedom and Trade Freedom – does seem to support the neo-liberal case. I will note, however, that Business Freedom is primarily determined by the speed/cost of starting, licensing, and closing a company. I will further note that this is heavily influenced by the efficiency of licensing agencies and the court system (especially bankruptcy courts). Again, these are measures of ‘Good Government.’ The neo-liberals do seem to have a case for lower trade barriers (‘Trade Freedom’), although a huge part of this correlation is driven by the fact that the entirety of Europe has relatively low trade barriers.

Conclusion

The Heritage Freedom Index is really a composite of measures that get at two different things: Good Government, and Less Government. Overall, the Good Government factors tend to dominate, and drive a lot of the correlation with good economic and quality of life outcomes. When one splits out the factors, the case for Less/Weaker Government weakens substantially, and the case for Clean/Non-Corrupt/Efficient government strengthens considerably.

This does not support many of the conclusions that are often drawn using the overall Heritage Index.