Where Hits Come From

Chris Dillow and James Surowiecki both weigh in this week on the subject of where hits come from. Surowiecki, as you might expect from a man who wrote a book called The Wisdom of Crowds, thinks that prediction markets are a good way of predicting hits. But he's fair to the other side, too:

The process of predicting whether a product will be a hit remains remarkably haphazard and erratic.
Many people argue that it’s foolish to expect otherwise, and that no science of success is possible. In the famous words of the screenwriter William Goldman, “Nobody knows anything.” The fate of a book or a movie, the argument goes, is determined by too many factors to be predictable—advertising, reviews, word of mouth, luck, and, in the case of big hits, a simple desire to see what all the fuss is about. De Vany, for instance, says that the box-office performance of Hollywood films is “chaotic” in the mathematical sense of the term. Three Columbia sociologists recently found something similar in a series of online experiments in which people were divided into eight groups, asked to listen to songs by unknown artists, rate them, and then decide which ones they’d like to download—after being told how often others had downloaded the songs. The highest-rated songs, it turned out, were not always the most frequently downloaded. And in each group a different song ended up topping the charts. In the laboratory, at least, success appeared to be essentially random.

Meanwhile, Dillow seems to think that success is impossible to predict and yet in some sense written in the stars:

A lot of success in any venture is impossible to predict. Many best-selling authors - including J.K. Rowling - were rejected many times by publishers. Even successful entrepreneurs reject profitable inventions. Decca famously turned down the Beatles...
How many great-selling books, popular bands or successful companies have we not had because they've been turned down? ...
If finance is easier and the costs of printing and marketing are lower, fewer potential successes should be filtered out. In this way, technical progress and easier finance can promote economic growth.

I'm not convinced by this. I don't think that JK Rowling would have been a success in all, or even most, worlds in which she got published. I think that success on her level is the result of a myriad of uncontrollable self-reinforcing mechanisms all coming together in the right place at the right time. Harry Potter books get read a lot because they're popular, rather than being popular because they're read a lot, if that makes any sense.

If that's the case, then Dillow's wrong to assume that the more books and movies get made, the more hits there will be. If anything, the opposite is the case: the world hasn't seen another band quite like the Beatles not because no subsequent band has had the same amount of talent but rather because the amount of choice facing consumers has increased so enormously. As a result, it's increasingly difficult for any artist or author to achieve lasting, broad-based success and acclaim. Even the longevity of Harry Potter is far from assured in the absence of hype about the next book or movie coming out: Harry Potter and the Chamber of Secrets has a sales rank of 118,787 on Amazon.

Now there are still hits, of course, and some of them are worth billions: Google springs to mind as one obvious example. But if Google hadn't been formed, other companies would have made almost as much money monetizing search. It's like the social value of Microsoft: to find that out, you have to compare the value of the world with Microsoft to the value of what the world would look like if Microsoft had never been formed. And that's a very different calculation to simply taking the world as it is and subtracting the value of Microsoft in this world.

To put it another way, all worlds have hits. And I don't think that the number of hits is correlated to how much technical progress or financial liquidity there is in any given world, even if many hits are the direct result of those two things.