Lies, damned lies and Web ad forecasts

It's pretty clear most forecasts, with some notable exceptions of course, suck. The entire endeavor is flawed. When predicting the future, most assume it will resemble the present and develop in a straight line. That doesn't always happen. (See Obama.) This is particularly true when it comes to industry forecasting. Internet research firms make a pretty penny peddling consulting services. One of their marketing strategies for the real dough of advising clients is to put out "industry forecasts." Reporters dutifully write stories with blaring headlines of giddy growth in mobile, seach, social media, whatever. These have long struck me as ridiculous. You're going to tell me what's going to happen in video advertising five years from now? What do they use a crystal ball? I recently asked an analyst about whether this stuff was just pure B.S. He claimed the forecasts are "directionally accurate." How accurate? "About 20 percent." Today I wondered how eMarketer could with a straight face forecast search ad spending in 2013. In the words of Lon Cohen:

The problem: these figures are used by companies to justify strategies to investors and the press. Yet nobody asks whether they're valid. For fun, I decided to hunt down an Internet ad forecast from years ago to see how the gurus did. Sure enough, Jupiter projected in August 1999 robust Internet ad growth, enough to reach $11.5 billion in 2003. Guess where it ended up? $7.3 billion, according to the IAB. That's not 20 percent off, it's 57 percent wrong. Jupiter assumed the past would keep repeating itself. It didn't factor in the structural weakness of the online bubble, along with the aftereffects of a recession exacerbated by a terrorist attack. Maybe it's not research firms' fault for not knowing the future, but it's everyone's fault who puts much stock in these forecasts, even "directionally."

Comments

[Michael Hubbard]

I totally disagree - I think forecasting is fantastic and accurate. As a matter of fact, I'm forecasting that you'll receive no less than 3,500 responses to this post that are in support of your view (give or take a few thousand).

[Niki Scevak]

Brian, great post and agree, although you picked a very exceptional year. From my time at Jupiter (I wasn't the guy who did the 99 forecast but I have done a few of them) and recording the accuracy (the firm did record the accuracy rate and it was always presented to prospective and current clients), the question was always one of time.

A one year forecast was very very accurate, a three year one was accurate (+/- 10%) and a five year accuracy rate was a fuzzy distribution (i.e. no correlation between right and wrong, but mostly right in general where wrong is off by 20%). Obviously the vintages varied and 99 was a bad one.

For instance, if you take the 2001 Jupiter ad forecast (http://www.jupiterresearch.com/bin/item.pl/research:vision/1215/id=89029), it was for $16.5bn in 2006, where the real number ended up being $16.9bn, according to the IAB.

But like everything else it was a question of best alternatives. Each company needs to make internal projections and assumptions around a host of things. No one knows for sure what is going to happen in the future but that doesn't mean people shouldn't make a best effort to quantify and plan what they think will happen.

And hence the market for industry/equity analysts making projections.

[Scott Rafer]

@niki that's not quite right. in the west, we always forecast up and ignore down cycles. in Asia, they tend to forecast up over a long period of time but with intra-decade volatility. we always ignore that volatility portion, to our peril.

[Brian Morrissey]

@niki: Long time not talk. Yes, Gary is on the hook for the '99 forecast. I cop to it being an extraordinary example. yet you choose a period of smooth expansion, equally unrepresentative of the up-and-down growth patterns Scott mentions. I don't see much evidence these forecasts take into account macroeconomic factors that clearly affect these comparatively small industries. For instance, what factors would be examined to come up with a forecast of search ad spending in 2013? How can these research firms, which to be fair mostly have analysts who are not trained economists, deign to put out a number that's labeled as anything more than a guess?

It's interesting that research firms put together these scorecards. i wonder why they don't publish them publicly. I'm going to take John Battelle up on his challenge of constructing one.

[kawika]

Brian, make your scorecard a wiki so readers can help you do the work. Sites like ZDNet's IT Facts (http://blogs.zdnet.com/ITFacts/) don't have the time to go back and check what's been posted, but there might be some fun predictions there with which you can seed your site.

[raivo pommer-eesti.www.google.ee.]

raivo pommer-www.google.ee
raimo1@hot.ee

Deflation hits Ireland

Ireland's consumer prices fell 2.6 per cent in March from a year ago, the sharpest rate of deflation since 1933, when the world was struggling through the Great Depression, official figures showed yesterday.

The March rate accelerated from an annual deflation rate of 1.7 per cent in February, the Central Statistics Office said. The report said there was no change in prices from February to March, which are now at August 2007 levels.

Ireland's deflation began in January and reflects the country's sudden fall into a deep recession.

The country last suffered from deflation in 1960.

Although lower prices can help spending and exports, deflation can be damaging for an economy if prices enter a downward spiral - consumers hold off buying items on expectations they will become cheaper, pushing retailers to cut prices to encourage spending, and so on.

Finance Minister Brian Lenihan, when announcing an emergency budget on Wednesday to trim 3.25 billion ($7.32 billion) from Ireland's ballooning deficit, said the Government expected deflation to average 4 per cent in 2009.