The new issue of the Financial Analysts Journal includes an article entitled “Buy-Side vs. Sell-Side Analysts’ Earnings Forecasts.”Financial Analysts Journal | Authored by Boris Groysberg, Paul Healy, and Craig Chapman, the article can be found in the July/August 2008 issue (Volume 64, Number 4) of FAJ, and could be accessed at the time of this writing through this free link on the CFA Institute publications site.  A related working paper that incorporates an analysis of the recommendations of a buy-side firm versus those of sell-side firms can be found on SSRN; this article focuses only on earnings-forecasting performance. The introductory summary includes this conclusion:  “The tests show that the buy-side analysts made more optimistic and less accurate forecasts than their counterparts on the sell side.”  I’m sure that members of the editorial board (and others) have examined the statistical analysis and pronounced it appropriate, so I’ll stick to the bottom line as reported.

The main issue that I have with the article is that when reading it, you have to continually remind yourself that the “buy-side analysts” that are talked about are from precisely one firm.  While the numbers may support the conclusion cited above for that one firm, it’s unwise to extrapolate from there.  The article says that “a replication of the tests on a broader sample would be interesting.”  I’d call them essential for making the case.  I’m sure that the anonymous firmThe buy-side firm was described in a couple of places as “top ten” in nature, although the way the text reads, that appears to be based on qualitative rankings (leaving open the question of how such rankings are derived) rather than by assets under management.  I did not attempt to identify it, although enough information about its attributes is given that a good guess could be made. that was analyzed is a good test case, but investment firms differ significantly one to another, and I don’t have any confidence that a broader analysis would produce the same conclusions.

Nonetheless, the article does document the relative forecasting errors of the buy-side firm versus a broad range of sell-side analysts, and explores several explanations for the subpar performance.  The list of possible reasons that were investigated include relative differences in the quality of analysts hired and retained, in the scope of analyst coverage, in the nature of the information sources available, in which stocks earnings estimates are produced for, and in methods of evaluating and incentivizing analysts.  For the firm in question, the authors found that the key elements of its relatively worse forecasting record were a higher retention rate for “low-quality analysts” and the lack of a systematic mechanism for tracking and evaluating its earnings estimates versus the Street.

All of those aspects of a firm need to be reviewed and parsed, so the analysis provides a good template to think about the structural elements that can contribute to a pattern of optimism.  Of course, a pattern of pessimism would be just as bad — we are looking for objectivity and accuracy — but in most forecasting endeavors, certainly in the investment world, it seems that the errors of optimism tend to predominate.  One critical element that should be included in a further analysis is a decomposition of the structure of the relative errors.  My guess is that there is a pattern of forecasting errors (and related trading decisions) that may explain a great deal about the firm.  Which “kinds” of stocks, with which attributes, and in which market environments, are the most susceptible to the errors?  Knowing that would provide valuable perspective.

It also may illuminate a source of optimism that is not postulated in the article.  One of the statistical tests showed that analysts that came from the sell side to the buy-side firm became more optimistic and less accurate after the switch.  Perhaps the nature of the firm is such that the analysts struggle with balancing their need for objectivity in their analysis with their desire to curry favor with portfolio managers, who have a big say in their compensation and career prospects, and who find it easier to justify their favorite positions if they are supported by above-consensus estimates.Of course, estimates different than consensus are to be encouraged, but the pattern of misses should not be one-sided.

The type of analysis that the authors have done is a critical step in understanding the efforts of an investment organization, and firms should be evaluating themselves on a variety of dimensions in that kind of systematic fashion.  While the article falls short of identifying a pattern of relative optimism at buy-side firms, it lays an important foundation for further work that should be of direct benefit to practitioners.