The problems of prophets and jesters

The need for more predictive intelligence is one that has seen a great deal of debate over the years. The first area of argument is as always (particularly when academics are involved) the definition of what prediction actually means, in the context of intelligence as both a process and as a product. (As much as we hate arguments over definitions, occasionally they ought to be revisited as first principles in a discussion, especially when a matter may be otherwise subject to misinterpretation.)

Our preferred view on this is that predictive intelligence means bounding the space of future uncertainties within an estimative framework. Good predictive intelligence therefore are estimates (and the tradecraft used to develop such estimates) that accurately, coherently, and pragmatically provide a view of bounded uncertainties that provide actionable insights to decision-makers that correspond closely to the actual course of future events. Good predictive intelligence also addresses the potential shocks - such as Black Swan events - that may emerge in future scenarios, in much the same way that well crafted capabilities intelligence addresses linchpins and milestones.

This is by no means an uncontroversial definition. There are those that would remove the term “predictive” entirely from the lexicon of intelligence, favoring only the specific verbiage of estimative intelligence. This we believe is a fallacy – first because the term is already in common use, formally or otherwise, and without seeking to distinguish good uses of the concept from those taught by false prophets one does a great disservice to those individuals which must work through the wider body of literature – or multiple agencies’ doctrines, where the concept may be favoured. The second reason we support discussion of predictive intelligence is because many intelligence consumers have articulated the need for improvement in the area as a key objective. There is certainly a common misunderstanding by consumers regarding the nature of what can be reasonably expected from prediction within intelligence, with the consumer’s desires leaning more towards the impossibilities of fortune telling. However, this makes it all the more critical that the purpose (and limitations) of predictive intelligence be communicated effectively to prevent such misunderstandings from colouring a consumer’s perceptions of products which are crafted to the best possible (realistic) standard – especially analysts are not issued a crystal ball with which to meet unrealistic and Hollywood influenced standards.

We see no conflict with the classic view of estimative intelligence in this discussion (although in some circles, we acknowledge that we may be a distinct minority of this opinion). After all, no less a luminary than the esteemed Harold Ford wrote that the among the questions that estimative intelligence seeks to answer are “what trends seem likely for the future, and how those trends might be affected in the event certain contingent events should occur” and that “the purpose, character, and significance of these courageous estimates of future unknowns has been recognized by many observers.”(The quotes are taken from his 1993 AFIO monograph on the topic, for those keeping score.) This very clearly refers to predictive intelligence in the same fashion that we describe it.

In a way, the debate over terminology and concepts – and in reality, the underlying purpose of what intelligence should seek to be – reminds us of the same debate over whether or not intelligence professionals should be responsible for examining questions of adversary intentions. While that debate has largely been settled conclusively in favour of that purpose, it was not always so. A good deal of literature – particularly that written in the earlier Cold War military context – made many of the same kinds of arguments regarding the impossibility of divining intention as we hear made regarding the prediction of future uncertainties. (And we should note that we still occasionally hear the arguments regarding intelligence on intentions when talking with law enforcement folks or others outside of the community.)

Having spent the foregoing establishing context, we recently also encountered a post by Charles Stross, one of our favourite jesters from the futurist court, which discussed the increasing difficulties of understanding technological drivers in out-years predictive scenarios given the accelerating pace of change (and adoption of that change). The points is well made by a chart taken from the Economist, depicting the deltas of technology penetration throughout history.

While technology drivers are often overstated in many futures intelligence exercises – particularly those conducted by individuals with their own stake in a given development or industry sector – there is no denying that from the perspective of certain intelligence accounts technology is often the defining feature around which other social, political, economic, and military events develop.

And it is not merely the rate of adoption within general societies that must be considered by intelligence professionals seeking a greater level of predictive analysis. The pace of hostile innovation has also radically accelerated, particularly when it comes to adoption of new technologies that enable asymmetric engagement, and which support the resilience of non-state actors under intense selection pressures. Many of these innovations are decidedly less than high tech – but as little as a decade ago still would have been the stuff of science fiction and laughed out of the briefing room had any intelligence analyst been foresighted (and naively foolish) enough to raise them as potential issues. We would do well to ensure that our current analytic environments do not likewise encourage such a narrow minded focus that would miss the sweeping rate of change that is bearing down on us, even as ridiculous as any given manifestation sometimes may seem from our current vantage point.

This is one of the reasons we seek to encourage the jesters, and to exhort the courtiers and fops to admit a bit more levity into their dance. For somewhere in the scullery there is a hard working young analyst that listens, and nurtures their own private vision of a future that may well be more probable than any included in the official powerpoint decks. If that analyst does not come forward for fear of the reaction within his shop’s environment, or is not given the opportunity to cultivate and explore those ideas, the loss of that concept may well contain the seeds of the next failure of imagination.

The perils of arbitrary and false precision

We find quite unhelpful the recent academic obsessions over estimative language – largely an exercise in the introduction of a numerical system which offers a degree of apparently comforting but entirely arbitrary, and therefore utterly false, precision. It seems however that we are in one of those cycles which seem to come along in the intelligence community every few decades or so, in which the numerologists and other soothsayers attempt to reshape the profession into their own desires for a more “scientific” practice.

Let us be clear. There are times when quantitative analytic methodology is vital – but there are far more situations in which it is misapplied, misunderstood, and entirely out of place. The latter comprise the vast majority of scenarios in which analytic tradecraft is called upon – not the least of which may be attributed to the highly unbounded and indeterminate nature of the problems with which we must grapple. And any time in which a quantitative basis has not been established, the insertion of numerical percentages for predictive purposes is little more than a farcical exercise in arbitrary selection. Over time, you may attune a group sufficiently in order to calibrate its judgment of these percentages in such a way as to create a consistency within that shared hallucination. However, this does not alter the underlying fallacy upon which such a house of cards is built. This is clearly shown in the number of cases in which the naive predictor is a better estimate of potential than the much vaunted group of experts’ judgment. Thus even in finance, the most precise of arenas, built upon the foundation of values, you will find predictions expressed equally alongside hedges – and the market littered with those who have failed to impose arbitrary figures on a highly indeterminate problem.

One of the greatest challenges in intelligence analysis is to understand the limits of prediction when going about the hard business of estimation. That understanding should shape the analyst’s focus on what ought to be examined for predictive possibility. These are, properly: the scope and nature of trends, drivers, and future scenario outcomes – and not the capricious shadings of difference between mathematical expressions of probability.

Estimative language has not been expressed through probability percentages for the sixty plus years of the intelligence community’s modern incarnation for good and well contemplated reasons. While the abstraction of the clean and sterile realm of mathematics is often a welcome change from the messy and hard realities of intelligence, that abstraction too frequently is used as a shield and an intellectual refuge for those unable or unwilling to embrace the challenge of actually doing intel.

Scientisim in intelligence analysis is a particularly seductive heresy. It offers the false promise of greater insight, should only additional efforts be applied more systematically, more rigorously, or with more and better data. But it has not been given unto us to see the future – no matter how carefully we might craft our equations. We may simply chart the boundaries of its outlines, and discuss the implications within the uncertainty space so described.

We have no doubt that we will revisit this discussion in short order. For now, however, we would close with an excellent reminder of the vast gulf of differences that may be concealed with that change of a single degree of significance in numerical expression. Originally produced for IBM, this admittedly dated video still serves to explain the staggering concepts of scale in a world of large numbers. (h/t to Thoughts Illustrated for pointing out its online incarnation.)