To assess the quality of an investment organization and its output, a certain sleuthing is required.  It doesn’t really matter whether you are looking at an asset manager or a research provider or a pension fund or whatever.  While the elements of the evaluation will differ somewhat — as will the relative importance of each in the final analysis — some basic principles apply.  First and foremost for me is the need to understand the organization and what it is trying to do.

In cases where you are asked to improve the structure and processes of a firm, there is a temptation to start giving recommendations before that understanding is achieved.  Even in situations where your evaluations are simply descriptive, the same pitfalls exist; you can start grading before the full picture is revealed.

To analyze the independent research firms contending for business under the Global Research Analyst Settlement,the research puzzle | This is the fourth in a series on the settlement.  This link will take you to an index of the postings on the topic that will include future postings as they are written. I asked them to complete an exhaustive request for proposal.  That allowed me to get as much information as I could to narrow the field a bit, since it was impractical to visit all of the research firms that sought to participate.  A similar approach is used in other parts of the business, especially in the selection of asset managers, and the process brings with it some common mistakes, mainly in relation to judgments about “performance.”  (More on that in a future posting.)

Onsite interviews are the real meat of the due diligence process, and there is no getting around spending the time and money it takes to do it well.Advisor Perspectives | This is a piece I wrote about the difficulties of doing due diligence “from a distance.” Once at those meetings, it is nothing fancy; just a pad of paper and some good questions.  Think of it as akin to the many “procedurals” on television, only more boring.  Piece by piece, information is revealed and put in context, as the mosaic is built, with that elusive understanding of what makes a firm work (or not work) the ultimate goal.

There are “aha” moments along the way (sort of like those that bring on the far-off stare of Gregory House), and they are very often triggered by little things.the research puzzle | My very first posting on this blog was called “look at the little things.” In the same way, you can sometimes learn more about how a firm works from a junior analyst than from the chief investment officer (to say nothing about those that are purely interested in the message control of marketing).  I have no interest in the “party line” other than to judge how far off of it the firm tends to drift.  That makes PowerPoint presentations and lots of people around a table unlikely venues for discovery.  One-on-one interactions with people in various parts of the process are always the most illuminating part of due diligence.

Let me be clear that it isn’t because people say things that they shouldn’t say, thereby revealing deep, dark secrets.  That’s not the case.  Rather, the interviews flow in different ways, people have a variety of perspectives, and important things can take on divergent looks “at altitude” versus on the ground.  After a cluster of such interviews, the result is a richer mix of inputs that tends to spawn a subsequent round of better questions.  The process is repeated until you attain the understanding you seek or realize that you are never going to get there.

The circumstances of the settlement were such that the providers of research wanted the business badly enough to be very forthcoming with detailed information and arranged interviews with anyone that I wanted to see.  (That isn’t always the case for other due diligence assignments, which makes it hard to endorse the firms being scrutinized without lots of caveats and reservations.)  In the next chapter of this series, I’ll detail some of my observations from those due diligence meetings.