Wednesday, August 14th, 2019
the future of analysis

In a previous posting,the research puzzle | The posting, “mind the gaps,” was written after my attendance (and presentation) at the 2019 CFA Institute annual conference in London. I wrote that when it comes to manager research, one “field of opportunity is the broad category of intelligence analysis, including innovations and improvements in the processes of investigation and decision making.”  The same statement could be applied just as easily to other functions within the investment world.

I was reminded of it when reading “The Future of Analysis,”CIA | The paper may be downloaded here. which was written by Joseph Gartin, the deputy associate director of learning for the Central Intelligence Agency.  It opens with this sentence, “The field of intelligence analysis is at an inflection point.”

As I read it, I was struck by how well Gartin’s paper — about “providing an advantage in the planning and execution of national security strategy” — applies to the research stage of the investment process.  As with intelligence analysis, much investment analysis still “comes down to this: read stuff, write stuff.”

The first part of the article provides historical context about the intelligence process, especially as related to the work of Sherman Kent.  He “surrounded himself with men of similar backgrounds, from a handful of prestigious universities and shaped by the seminal events of the early 20th century.  All of them were very good at the same thing: reading stuff, writing stuff.”  But subsequent analytical failures “have highlighted the inherent weakness of simply relying on very smart (and very similar) people to read a lot of reports and make accurate assessments.”  Sound familiar?

What had emerged was a “dominant analytic culture,” but Gartin asserts that “it is clear the business of analysis is in flux,” and mentions five drivers of change (each of which I will comment upon in relation to investment organizations):

1) Structured analytic techniques.  Gartin quotes Richards Heuer’s seminal work, Psychology of Intelligence Analysis,CIA | It is also available on the CIA’s site. “Intelligence analysts should be self-conscious about their reasoning processes.  They should think about how they make judgments and reach conclusions, not just about the judgments and conclusions themselves.”  Trying to get investment professionals to talk about the “how” rather than the “what” is very difficult, and most organizations get failing marks on this point.  Changes in culture, talent, and training are needed.

2) Advances in cognitive sciences.  Gartin states that intelligence analysts need to “better understand how actors make decisions, how badly humans gauge risk and reward, and how we conflate probability with confidence.”  Ditto for investment analysts, so we should build processes and guardrails into what we do to provide checks on the well-documented human tendencies that lead to poor decision making.

3) Improvements in quantitative forecasting.  The author cites Philip Tetlock and says that a reliance on the structured analytic techniques mentioned above “does not necessarily produce better results.”  Forecasting is a problematic activity in many investment processes, with undocumented (and therefore untracked) predictions — ranging from educated guesses to gut feels to thinly-veiled hopes — driving decisions, without much reflection on their nature or worth.  Both qualitative and quantitative forecasting methods need to be improved.

4) The “hyper-specific” analytical response to a traumatic event.  In the case of intelligence analysis the event was 9/11; for investment organizations, it was the financial crisis.  That experience led to significant changes in policy, behavior, and investing techniques.  Confidence has been rising over time that the quantitative risk management tools put in place in response to that trauma will work as promised, but they haven’t had a big test yet.  We’ll see what happens in a more wicked environment.

5) The explosion of data.  “We are awash in ones and zeroes that can be linked, analyzed, and leveraged, if we ask the right questions of the right data sets.”  Gartin writes that “analysis today encompasses targeting, full-motion remote sensing, financial intelligence, identity intelligence, structured observation management, prediction markets, financial intelligence, activity-based intelligence, data analysis, object-based production, cyber forensics, social media analysis, and more.  The list is dizzying.”  True for us too.

Looking forward, given expanded technology capabilities, analysts need to be “focused on work that is consistently higher on the value chain,” creating job responsibilities that look much different than those of the past.  (You should take the time to read the short section of the paper that provides the advice offered by a digital assistant to an analyst.  It will open your eyes to some of the possibilities.)

To close, Gartin offers “things we can do today to help us shape the future of analysis,” which apply to the craft in the investment realm as much as the intelligence one.  As important is the realization that the disruptive advancements that have been witnessed in other areas know no boundaries.

“Analysis will not be immune.”