The Uselessness of Zagat's Cost Estimates

Eleven Madison Park is a very good, and very expensive, restaurant. At dinner, the cheapest menu costs $82 per person. A glass of red wine will cost you somewhere between $8 and $26. So a dinner with one drink can't cost less than $90 before tax. Add tax, and you get to $97.54. According to Zagat, the average tip in New York is 19%. Apply that to the pre-tax total, and you get a minimum total bill including food, one drink, tax and tip of $114.64.

Yet according to latest issue of Zagat, the "average estimated cost" of a meal with one drink, tax, and tip at 11 Madison Park is just $104. Which isn't even possible, unless you're tipping just 7.1%.

Now here's the really crazy thing: the 2008 Zagat guide is, in this respect, a vast improvement over the 2007 Zagat guide, which ludicrously maintained that the average cost of a meal at 11 Madison Park was just $66. Perhaps the Zagats took Steve Cuozzo's criticisms to heart and vowed to do better this year. But undeniably the $104 figure, while still on the cheap side, is a lot closer to reality than the $66 figure was.

So one cheer to Zagat's for slowly moving away from its notorious underpricing. But no cheers at all for its press releases and WSJ op-ed which talk at length about inflation in the cost of fine dining. The average restaurant meal has not really increased in price over the past year, they say, although there has been more inflation at the top end of the scale. Well yes, if you really think that the price of a meal at 11 Madison Park has gone up by more than 57% in the space of one year, then I suppose you could demonstrate a certain amount of price inflation. But really the Zagats aren't measuring meal-price inflation at all, they're just measuring changes in the entirely fictional amounts of money which appear in their guide. No one actually believes them:

Overall dining prices were only three cents more than last year, at $39.46 per dinner. That’s such a low figure that we guess they must be averaging Gray’s Papaya into it.

So there's no need for someone like Barry Ritholtz to tie himself up in knots trying to work out how the numbers could possibly be true. Yes, small-plate menus are becoming more common, but no, they don't bring prices down; rather, they bring prices up. And yes, prix-fixe dinner menus are also becoming more common, but in general they cost more than one would expect to pay a la carte, not less.

The simple fact is that there is absolutely no reason why anybody should pay any attention at all to the prices in the Zagat guide. And the idea that prices in New York ($39.43, we're told) can somehow be usefully compared to prices in London ($78.57) is particularly laughable – the methodology for arriving at those prices is going to be very different from city to city, as is the ratio of high-end to low-end restaurants being rated.

Zagat's food ratings go up over time, and its cost estimates wander around randomly according to the whim of the guide's eponymous owners. Neither are remotely useful as emprical evidence of anything at all.