Why Report Unpublished Findings?

Stephen Dubner at Freakonomics asks a very important question:

Is it my imagination, or is there more attention than ever being paid to the papers presented at econ conferences? Is this a good thing (more information) or a bad thing (the information isn't ready for prime time yet)? And why is this happening?

Since this largely an economics research blog, I grapple with Dubner's question all the time.

Publications like NYT, WSJ, FT, and Slate have reported on non-peer reviewed research for years -- so this phenomenon is not new -- but as with most other beats in the new-media age, coverage of econ research has definitely intensified. (And it's also ironic that Dubner is asking the question, since the success of Freakonomics the book has played a large part in spurring the extra interest.)

The problem in the case of economics is that research that might have something relevant to say today can take months or years to make it into a peer-reviewed journal. And many within the discipline complain that the delay isn't because the original research was faulty but because of the politics of the peer-review system.

Institutions like the National Bureau of Economic Research (NBER) have in some sense taken over for the peer-review system. Here is what the NBER's Martin Feldstein told me in September:

...if you look at academic papers you see that many of them cite the bureau working paper even if it's subsequently got published because they didn't bother to follow up and see it.

Now, what peer review does is to say is that this paper, especially if it's in a good journal, is a quality piece of research. Being in the bureau working paper series is another kind of certification. It says you may not know professor Smith or assistant professor Smith, but he or she must be pretty good or they wouldn't be part of the NBER family, and so the quality control comes in that way.

Meanwhile, the representation of economists from elite universities in the top journals has fallen thanks to the internet where ideas can get near-instantaneous coverage.

My own view is that it's a necessity to cover unpublished research these days because it's already impacting policy decisions.

The first case that comes to mind is New York City's move to appoint Harvard's Roland G. Fryer as its Chief Equality Officer.

As C.E.O., he's in charge of a pilot program in the city with the country's largest public school system that will pay students for getting better grades.

And how many papers has Fryer published on the efficacy of such programs? Zero.