Economics: Modern-day 'econometrics' have got the forecasts wrong

This has been a tough recession. Many businesses and households are still trying to cope with the legacy of the biggest drop in GDP in the UK since 1945: unemployment has taken a heavy toll and government finances are in such a parlous condition that they will take years to repair. And another much smaller group had a severe battering, but this has been more in terms of credibility than money. Very few economists predicted the recession two years ago and, of those that did, very few expected a downturn of the length and depth from which we are slowly emerging. Given the alleged sophistication of forecasting techniques and developments in computing, how can a collective failure of some of the sharpest analytical minds in the country be explained?

The extent to which the economics ‘profession’ was wide of the mark is apparent from a monthly report produced by HM Treasury. This compares the forecasts of the key indicators of 41 major organisations, such as banks, consultancies, academic institutions and international agencies. The January 2009 report showed that the consensus (the median) forecast was that GDP would fall over the coming 12 months by 2.4%, that unemployment would climb to 1.9 million by year-end and the government finances would be in debt to the tune of £125bn.

The actual numbers were quite different: activity fell twice as much (by 4.8%) and unemployment ‘only’ reached 1.6 million - but government finances were more than £50bn (40%) deeper in the red.

Just as worrying was the difference between the experts themselves at the time. In January 2009, the range of GDP forecasts was from a high -0.5% to a low of -3.2%; for consumer spending, the spread was +0.5% to -3.6% and of export growth from +3.5% to -7.9% (it turned out to be nearer 11%). How can any business plan on the basis of such data? Looking at the numbers justifies the old claim that God created economists to make astrologers look credible.

Part of the problem lies in the alleged ‘sophistication’ of economists these days. Older hands (or cynics) would say that economics today is all about mathematical formulae rather than judgements on the real world, the belief that events and the economy can be modelled. All the recession has shown is that these wonderful ‘econometric’ techniques have helped economists to be wrong with more accuracy, rather than they are right. While models undoubtedly ensure internal consistency, (as in, that the numbers add up to 100), they have proved woefully inadequate as forecasting tools.

Models depend on the statistical relationships between the key variables, which in turn depend on the past being a good guide to the future as well as on the reliability of the raw data. The confidence in model-based forecasts is diminished on both counts. The data problem has been well documented and accusing fingers are often pointed at the Office for National Statistics, which constantly revises many of the key numbers long after the first estimates are published. It has been said with some justification that government statistics are in such a ropey state that there is now more uncertainty about the past than there is about the future. Clearly, if you don’t know where you are starting from, it is hard to plot the forward journey.

And the world does change. Assuming that past relationships hold can lead economists astray while not spotting that a particular situation is virtually unprecedented is a cardinal error of judgement. So much about the past 18 months was new that the old rules should have been dumped and any computer-generated forecast should have been over-ridden by judgement. Had that happened, forecasts might have been better ­ but it would have raised the question of why such models are used in the first place. A real Pandora’s Box.

Economists have brought much of this confusion on themselves. Rather than try to convince people that they can forecast to within three places of a decimal point, they should own up and focus on what can be done. Nobody knows (and still fewer care) about the difference between 2.8% and 2.9% and investing time and resources to justify a particular number is a huge waste. What matters is if the new number is higher or lower, faster or slower, better or worse and some broad estimate of the extent of the difference. Highlighting the vulnerabilities, such as the impact of the debt overhang in the consumer sector, or establishing upper and lower ranges ­ ‘scenarios’ ­ are much more helpful than point forecasts.

Just as economists have claimed too much for themselves, so many companies and policymakers have expected too much of economists, or at least asked the wrong questions. Perhaps this recession will lead to a reality check on all sides, or some appreciation of what can reasonably be done. Denis Healey, Labour chancellor in the 1970s, once famously remarked that he wanted to do for economic forecasters what the Boston Strangler did for door-to-door salesmen. This might now happen as a result of the recession.

Dennis Turner is chief economist at HSBC