“THE forecaster is like an entrepreneur,” says Roman Frydman. “He uses quantitative methods, but he also studies history, and relies on intuition and judgment. He is not a scientist.” According to the New York University economist, this fact has been lost on contemporary economists, who continue to pursue the perfect economic forecast despite abundant evidence that it does not, and cannot, exist. They dismiss their repeated failures in much the same way that self-styled reformers in Mr Frydman's native Poland once insisted that socialism was great, but just needed to be carried out better.

In the economics profession the leading inheritors of this communistic mindset, says Mr Frydman, are the proponents of rational-expectations theory, which assumes that the economy and the individuals within it act with perfect foresight. Yet he is equally critical of the more fashionable school of behavioural economics, or at least those of its practitioners who claim that although people are irrational, their irrationality can be modelled so precisely that the future can be forecast with great precision.

In a new book, “Imperfect Knowledge Economics”, written with Michael Goldberg of the University of New Hampshire, Mr Frydman sets out an alternative approach to prediction, in which the forecaster recognises that his model will inevitably be less than perfect. Their work has received glowing praise from Nobel-prize-winning economists such as Kenneth Arrow and Edmund Phelps, who wrote the introduction to the book—though it is unlikely to have gone down so well with Robert Lucas, who won the Nobel for his work on rational expectations.

There is nothing new in economics about the idea that people must make decisions based on imperfect knowledge. Frank Knight gave his name to “Knightian uncertainty” thanks to his 1921 book, “Risk, Uncertainty and Profit”, which noted that most business decisions involve a step into an unknown that is to some degree unmeasurable. And John Maynard Keynes observed that “human decisions affecting the future, whether personal or political or economic, cannot depend on strict mathematical expectation, since the basis for making such calculations does not exist.”

While reflecting these insights, imperfect-knowledge economics still sees a role for economic theory in forecasting. Messrs Frydman and Goldberg argue that, to be useful, economic forecasting models should be based on qualitative regularities in the way that market participants respond to new information—that is, patterns of behaviour that are observable and somewhat predictable. Though not perfect, these will often give a better clue to the future than no model at all, or models based on rational expectations or behavioural economics.

Take bulls and bears, for example. Their analysis of the fundamentals leads them to opposite conclusions about where prices are going. But there is evidence that the way they revise their forecasts in the light of price movements may share common features, such as a tendency to become more risk averse the further the price of an asset moves away from what is generally believed to be its long-term fundamental value. This may work eventually to return the asset price to its fundamental value, though it may also cause it to deviate significantly from this value for long periods of time. This approach will not generate the “sharp predictions” beloved of most contemporary economists—which are doomed by imperfect knowledge to be wrong. But it will provide a broad sense of the state of play, which an enterprising forecaster can usefully combine with experience, intuition and so forth when making a decision.

To show how this works, Messrs Frydman and Goldberg examine the persistent failure of economists to predict movements in the currency markets. According to Kenneth Rogoff, an economist at Harvard who has long attempted to find rational models for predicting currency fluctuations, “it is stunning how hard it is to explain movements in exchange rates.” All the models based on rational expectations now say that, on fundamentals, the euro is overvalued against the dollar, he reckons. But does that mean the dollar will soon rise? Mr Rogoff says he has no idea.

In rational-expectations theory, a range of variables including inflation, interest rates and growth should have a predictable impact on currency movements, but in practice this theory has proved less useful for forecasting than tossing a coin. Among rational economists, the debate is over “whether the glass is 5% full or 95% empty,” he says. Only over longer periods—say two to four years—is there any evidence of exchange-rate predictability, which is far too long to be useful to traders or policymakers.

By contrast, the model developed by Messrs Frydman and Goldman, which assumes imperfect knowledge and learning, does significantly better than tossing a coin, although it is by no means right all the time. Mr Rogoff describes this work as innovative. Now, however, it must demonstrate that it can consistently maintain explanatory power in the future and over a range of markets, he believes.

Maths lesson

Messrs Frydman and Goldberg are now turning their attention to the troubled subprime-mortgage markets, and the performance of the rating agencies. The rating agencies, argues Mr Frydman, have generally been better at rating corporate bonds than rating asset-backed collateralised-debt obligations. Why? One reason is that the rating agencies used both a mathematical model and the judgment of their in-house specialists when forecasting the default probabilities of corporate bonds; on subprime-related securities, they could only use mathematical models, not least because the instruments were so new. “They had no experience, no intuition, no entrepreneur,” he says. That is “empirical proof that relying on models alone is not wise.”