Ranking Fed Forecasters The Wall Street Journal examined more than 700 predictions made between 2009 and 2012 in speeches and congressional testimony by 14 Fed policymakers. See individual scores and compare officials’ comments against each other. (Read full methodology.)

METHODOLOGY

To rank the accuracy of forecasts by Federal Reserve officials, The Wall Street Journal searched more than 700 speeches and testimonies made since the end of the recession in June 2009, looking for forward-looking statements on inflation, unemployment and growth.

The statements included specific forecasts—“I expect the unemployment rate will fall to around 8% to 8.5% by the end of the year”—as well as general ones—“The recovery is likely to be painfully slow,” for example.

Each statement was scored on a scale from -1, far from what actually happened, to +1, very accurate. To calibrate the scoring, the Journal used a statistical tool known as standard deviation, which is a measure of the normal variability in a collection of data. If a forecast was within one standard deviation of what is considered normal variation, it was considered correct and awarded one point. If it was more than one standard deviation away, it was considered incorrect and lost a point.

Officials were awarded half credit when they made forecasts that were in the right direction, but lacked precision. The benchmarks used were gross domestic product for growth, the unemployment rate and the change in Commerce Department’s personal consumption expenditure price index for inflation.

For example, in January 2011, Boston Fed President Eric Rosengren said, “My own forecast is for growth of 3.5% to 4.0% over 2011.” He received a mark of -1 because growth was 2% that year and the annual standard deviation of growth over the past two decades was 0.68 percentage points, making him more than one standard deviation off.

In October 2011, Cleveland Fed President Sandra Pianalto said, “I don’t expect the pace of growth to pick up very soon; my outlook for real GDP growth in 2012 is about 2%..” She got a full point for that forecast because growth was 1.7%, within one standard deviation of her projection.

Some comments were too vague to score. Comments that contained a mix of correct and incorrect predictions received a score of zero.

The Journal calculated the average—using total points and the number of predictions—to produce scores. A score of 1 represents nearly perfect foresight; -1 reflects a record that is always wrong; 0.5, consistently in the right direction; -0.5, consistently in the wrong direction. Zero means forecasts were as often correct as incorrect.

A specific forecast that was repeated between meetings of the Federal Open Market Committee meetings was counted only once.

Moody’s Economy.com, a private forecasting firm, checked the Journal’s scoring. The final results by the Journal reflect some suggestions by Moody’s.



— Jon Hilsenrath and Kristina Peterson

Note: Mr. Bullard often gives PowerPoint presentations without prepared text. Scores based on press releases with those presentations.



Source: WSJ analysis of more than 700 Federal Reserve officials’ speeches and testimony from June 2009 through December 2012