Economics: Modern-day 'econometrics' have got the forecasts wrong

This has been a tough recession. Many businesses and
households are still trying to cope with the legacy of the biggest drop in GDP
in the UK since 1945: unemployment has taken a heavy toll and government
finances are in such a parlous condition that they will take years to repair.
And another much smaller group had a severe battering, but this has been more in
terms of credibility than money. Very few economists predicted the recession two
years ago and, of those that did, very few expected a downturn of the length and
depth from which we are slowly emerging. Given the alleged sophistication of
forecasting techniques and developments in computing, how can a collective
failure of some of the sharpest analytical minds in the country be explained?

The extent to which the economics ‘profession’ was wide of the mark is
apparent from a monthly report produced by HM Treasury. This compares the
forecasts of the key indicators of 41 major organisations, such as banks,
consultancies, academic institutions and international agencies. The January
2009 report showed that the consensus (the median) forecast was that GDP would
fall over the coming 12 months by 2.4%, that unemployment would climb to 1.9
million by year-end and the government finances would be in debt to the tune of
£125bn.

The actual numbers were quite different: activity fell twice as much (by
4.8%) and unemployment ‘only’ reached 1.6 million – but government finances were
more than £50bn (40%) deeper in the red.

Just as worrying was the difference between the experts themselves at the
time. In January 2009, the range of GDP forecasts was from a high -0.5% to a low
of -3.2%; for consumer spending, the spread was +0.5% to -3.6% and of export
growth from +3.5% to -7.9% (it turned out to be nearer 11%). How can any
business plan on the basis of such data? Looking at the numbers justifies the
old claim that God created economists to make astrologers look credible.

Part of the problem lies in the alleged ‘sophistication’ of economists these
days. Older hands (or cynics) would say that economics today is all about
mathematical formulae rather than judgements on the real world, the belief that
events and the economy can be modelled. All the recession has shown is that
these wonderful ‘econometric’ techniques have helped economists to be wrong with
more accuracy, rather than they are right. While models undoubtedly ensure
internal consistency, (as in, that the numbers add up to 100), they have proved
woefully inadequate as forecasting tools.

Models depend on the statistical relationships between the key variables,
which in turn depend on the past being a good guide to the future as well as on
the reliability of the raw data. The confidence in model-based forecasts is
diminished on both counts. The data problem has been well documented and
accusing fingers are often pointed at the Office for National Statistics, which
constantly revises many of the key numbers long after the first estimates are
published. It has been said with some justification that government statistics
are in such a ropey state that there is now more uncertainty about the past than
there is about the future. Clearly, if you don’t know where you are starting
from, it is hard to plot the forward journey.

And the world does change. Assuming that past relationships hold can lead
economists astray while not spotting that a particular situation is virtually
unprecedented is a cardinal error of judgement. So much about the past 18 months
was new that the old rules should have been dumped and any computer-generated
forecast should have been over-ridden by judgement. Had that happened, forecasts
might have been better ­ but it would have raised the question of why such
models are used in the first place. A real Pandora’s Box.

Economists have brought much of this confusion on themselves. Rather than try
to convince people that they can forecast to within three places of a decimal
point, they should own up and focus on what can be done. Nobody knows (and still
fewer care) about the difference between 2.8% and 2.9% and investing time and
resources to justify a particular number is a huge waste. What matters is if the
new number is higher or lower, faster or slower, better or worse and some broad
estimate of the extent of the difference. Highlighting the vulnerabilities, such
as the impact of the debt overhang in the consumer sector, or establishing upper
and lower ranges ­ ‘scenarios’ ­ are much more helpful than point forecasts.

Just as economists have claimed too much for themselves, so many companies
and policymakers have expected too much of economists, or at least asked the
wrong questions. Perhaps this recession will lead to a reality check on all
sides, or some appreciation of what can reasonably be done. Denis Healey, Labour
chancellor in the 1970s, once famously remarked that he wanted to do for
economic forecasters what the Boston Strangler did for door-to-door salesmen.
This might now happen as a result of the recession.

Dennis Turner is chief economist at HSBC