Insight problems, red cell mindsets and alternative analysis

We have long held mixed opinions regarding the computer security guru Bruce Schneier. While he often says interesting and provocative things, and has a distinct flair for memorably naming common phenomena (including introducing nomenclature such as security theatre, which even senior Transportation Security Administration officials have adopted in their own verbiage). At the same time, his off the cuff commentary frequently reaches far beyond his area of expertise into things of which he clearly has limited knowledge, but which he asserts with the same confidence – with less than useful results. It is a classic problem of the expert’s paradox, one frequently seen in those SME’s that spend a great deal of time in the media’s limelight.

Despite this caveat, we do commend to our readers a recent piece in which Schneier has brought to our attention an interesting course in computer security. The course attempts to inculcate the “attacker’s mindset” into new students, teaching them to view problems from the adversary’s perspective in what intelligence professionals will recognize as a classic red cell fashion. He notes that this kind of thinking is quite alien to most engineers. We concur, and to this category we would also add lawyers, most economists and political scientists, as well as others of like inclination which have been educated within the formal strictures of similar academic disciplines that do not value alternative models outside of their own recognized boundaries.

We are happy to see such matters being discussed in the otherwise normally disconnected halls of the academy. We feel it crystallizes an approach to addressing one of the core problems of the intelligence profession – that of teaching analysts about insight problems, and in particular the kind of insight problems that require experiential epiphanies to begin to understand. Much of the lack of creativity and loss of imagination in the intelligence field can be attributed to attempts to bound non-deterministic problems too tightly within the confines of a given methodological approach. While structured analytical techniques are vital to exploring the fleeting quicksilver of insight, those who try to squeeze too hard will find that quicksilver escapes their grasp. You cannot teach insight – you must inspire it, and teach the methods which can reliably generate such inspiration.

We view this as a vitally important and almost entirely neglected aspect of current intelligence education and training. Given that alternative analysis has been enshrined as a requirement to meet community standards, and that formal red cell efforts continue to proliferate throughout many agencies and organizations, cultivating the kind of analysts which can perform well in those environments is vital. And unfortunately, most current instruction falls woefully short of that which is needed to accomplish such a task.

The problems of prophets and jesters

The need for more predictive intelligence is one that has seen a great deal of debate over the years. The first area of argument is as always (particularly when academics are involved) the definition of what prediction actually means, in the context of intelligence as both a process and as a product. (As much as we hate arguments over definitions, occasionally they ought to be revisited as first principles in a discussion, especially when a matter may be otherwise subject to misinterpretation.)

Our preferred view on this is that predictive intelligence means bounding the space of future uncertainties within an estimative framework. Good predictive intelligence therefore are estimates (and the tradecraft used to develop such estimates) that accurately, coherently, and pragmatically provide a view of bounded uncertainties that provide actionable insights to decision-makers that correspond closely to the actual course of future events. Good predictive intelligence also addresses the potential shocks - such as Black Swan events - that may emerge in future scenarios, in much the same way that well crafted capabilities intelligence addresses linchpins and milestones.

This is by no means an uncontroversial definition. There are those that would remove the term “predictive” entirely from the lexicon of intelligence, favoring only the specific verbiage of estimative intelligence. This we believe is a fallacy – first because the term is already in common use, formally or otherwise, and without seeking to distinguish good uses of the concept from those taught by false prophets one does a great disservice to those individuals which must work through the wider body of literature – or multiple agencies’ doctrines, where the concept may be favoured. The second reason we support discussion of predictive intelligence is because many intelligence consumers have articulated the need for improvement in the area as a key objective. There is certainly a common misunderstanding by consumers regarding the nature of what can be reasonably expected from prediction within intelligence, with the consumer’s desires leaning more towards the impossibilities of fortune telling. However, this makes it all the more critical that the purpose (and limitations) of predictive intelligence be communicated effectively to prevent such misunderstandings from colouring a consumer’s perceptions of products which are crafted to the best possible (realistic) standard – especially analysts are not issued a crystal ball with which to meet unrealistic and Hollywood influenced standards.

We see no conflict with the classic view of estimative intelligence in this discussion (although in some circles, we acknowledge that we may be a distinct minority of this opinion). After all, no less a luminary than the esteemed Harold Ford wrote that the among the questions that estimative intelligence seeks to answer are “what trends seem likely for the future, and how those trends might be affected in the event certain contingent events should occur” and that “the purpose, character, and significance of these courageous estimates of future unknowns has been recognized by many observers.”(The quotes are taken from his 1993 AFIO monograph on the topic, for those keeping score.) This very clearly refers to predictive intelligence in the same fashion that we describe it.

In a way, the debate over terminology and concepts – and in reality, the underlying purpose of what intelligence should seek to be – reminds us of the same debate over whether or not intelligence professionals should be responsible for examining questions of adversary intentions. While that debate has largely been settled conclusively in favour of that purpose, it was not always so. A good deal of literature – particularly that written in the earlier Cold War military context – made many of the same kinds of arguments regarding the impossibility of divining intention as we hear made regarding the prediction of future uncertainties. (And we should note that we still occasionally hear the arguments regarding intelligence on intentions when talking with law enforcement folks or others outside of the community.)

Having spent the foregoing establishing context, we recently also encountered a post by Charles Stross, one of our favourite jesters from the futurist court, which discussed the increasing difficulties of understanding technological drivers in out-years predictive scenarios given the accelerating pace of change (and adoption of that change). The points is well made by a chart taken from the Economist, depicting the deltas of technology penetration throughout history.

While technology drivers are often overstated in many futures intelligence exercises – particularly those conducted by individuals with their own stake in a given development or industry sector – there is no denying that from the perspective of certain intelligence accounts technology is often the defining feature around which other social, political, economic, and military events develop.

And it is not merely the rate of adoption within general societies that must be considered by intelligence professionals seeking a greater level of predictive analysis. The pace of hostile innovation has also radically accelerated, particularly when it comes to adoption of new technologies that enable asymmetric engagement, and which support the resilience of non-state actors under intense selection pressures. Many of these innovations are decidedly less than high tech – but as little as a decade ago still would have been the stuff of science fiction and laughed out of the briefing room had any intelligence analyst been foresighted (and naively foolish) enough to raise them as potential issues. We would do well to ensure that our current analytic environments do not likewise encourage such a narrow minded focus that would miss the sweeping rate of change that is bearing down on us, even as ridiculous as any given manifestation sometimes may seem from our current vantage point.

This is one of the reasons we seek to encourage the jesters, and to exhort the courtiers and fops to admit a bit more levity into their dance. For somewhere in the scullery there is a hard working young analyst that listens, and nurtures their own private vision of a future that may well be more probable than any included in the official powerpoint decks. If that analyst does not come forward for fear of the reaction within his shop’s environment, or is not given the opportunity to cultivate and explore those ideas, the loss of that concept may well contain the seeds of the next failure of imagination.

RAND views analytic tradecraft

The new RAND study “Assessing the Tradecraft of Intelligence Analysis” has been out for more than a few days now, but deserves an in depth look by those that may have merely given it a passing glance. It was brought to our attention by the Analyst’s Corner, which has become increasingly consistently interesting (although we knew it would be, given the earlier writings of its author.)

What is interesting is that the report is very much a snapshot of a transition period – one might even be temped to say one that was taken at the height of the revolution in intelligence affairs. We agree with our virtual colleague Michael Tanji in his statement that “The dominant pattern in the U.S. intelligence agencies has been not stasis but almost constant revision, even to the point of disruption.” It is for this reason we have tended to look upon the cottage industry of intelligence reform with great suspicion, as too often of late we have had more than our fill of academics and other outsiders writing in with inspiration from what those in forward deployed locations often call the good idea fairy.

However, RAND’s study brings to the debate a number of important concepts, that while not new, certainly need to be circulated more widely. In part, this is due to the commendable methodology chosen for the study, in conducting formalized interviews across the community, targeted against not merely the ever changing organization charts (which as RAND itself noted “names have been a moving target”, given reorganization), but against the National Intelligence Priorities Framework and the Analytic Resource Catalog.

Among these critical concepts are the emphasis that analytic tradecraft is about the management of tradeoffs. There are few other human endeavors where this is not true, but for too long the community has focused on the ideal state, rather than maximizing the best possible outcome from the existing states. The ideal picture approach is very much an academic conceit, and assumes a mythical power to create organizational change simply by redrawing organograms or renaming offices under some centralized directive from on high. The real community simply does not respond to such abstracts in the clean and dispassionate fashion that many reformists would wish for. These tradeoffs are also one of the reason initiatives which begin organically within the working level line analysis shops are the most successful, as they allow those with the greatest stake in the outcome to balance their tradeoffs to the best possible effect.

The RAND study addresses interesting aspects of the increasingly dominant focus on current intelligence at the expense of longer term deep analysis. It also touches upon the issues of compensation and human resources that we have so often mentioned in these pages. We are quite pleased to see an increasing recognition of the importance of targeting analysis as a distinct discipline within the field – and given the delay between the interviews and the release of the public paper – one that we feel has been increasingly internalized within the community.

Collaboration and data sharing issues are discussed, but fall far lower in the spectrum than discussions of intelligence quality and value – quite in line with our own experiences.

The need to strengthen analytic training and education throughout the community is likewise emphasized, with the idea of a standard curriculum model again surfacing. We are aware of at least one quite promising effort in that regard, that goes far beyond what is typical academic fare; and hope to see further aspects of the model developed for mid-level and journeyman class analysis audiences in future iterations.

All in all, the RAND study is an excellent contribution to the literature which we are grateful now sees the light of day. There is much food for thought, which we will no doubt revisit again in due course. We did initially give pause upon a day’s reflection, fearing our agreement with the paper stemmed too much from a potential echo chamber effect of seeing similar views reflected back at us. However, these are things that are rarely formally captured in discussions of reform or the future of intelligence (at least, those written by outsiders). It is important to get them onto the table in a more formal setting – for as much as we believe in the value of the blog, it is a different vehicle for inquiry and scholarship than that of a more rigorous study approach.

Wx-ing historical

We have addressed the recent fad towards addressing climate change as an intelligence issue several times over the course of this blog. We still remain convinced that while weather intelligence – Wx – will always remain a key factor in many accounts, climate change as a long term issue is simply beyond even the horizon that can be expected of the deepest of futures intelligence looks.

We are apparently in the minority in this view. We recently acquired a copy of an unclassified 1978 research paper from Central Intelligence Agency’s National Foreign Assessment Center (now republished by the University Press of the Pacific in 2005) which examined this very issue under the title “Relating Climate Change to its Effects”. For those younger analysts unfamiliar with the misty back history of old bureaucratic battles and therefore older acronyms, NFAC was the renamed Directorate of Intelligence (DI) under DCI Turner in 1977 – a designation which lasted only until reorganization under DCI Casey’s tenure in 1981.

We think perhaps this product might have best been buried with the old name. As a paper, it is almost entirely uninspiring – a mere 8 pages of substantive text, followed by hundreds of pages of tables and statistics that are the hard copy rendition of a contemporaneous data tape, mostly consisting of temperature and precipitation measurements assembled by a university contractor on behalf of USDA. These form the inputs to a simple climate model that was intended to provide for long term predictions of weather effects given specific outcomes, such as global cooling - a key concern of the day. (However, to their credit, the designers did examine the potential for global warming as well – which speaks well of the analytic rigour of the DI under any name, even if the paper is mute testament to just how badly scientific and technical experts can be at communicating with their readership through written intelligence products.) A speculative product such as this can be expected to offer no real conclusions – rather simply serving as a possible set of boundaries within the uncertainty space of future scenarios. However, it might well have made more explicit the effects it claimed to consider within the range of those scenarios.

But again, this serves to illustrate both the waste and the foolishness of attempting a futures intelligence estimate so far into the out years. We are not issued crystal balls when we are granted entry into the profession. It also serves to illustrate the perils of the arbitrary application of quantitative analysis as a fig leaf over unsustainable judgments. The model – no doubt painstakingly assembled and hard fought at the methodological level – is by its very nature the product of 1970’s era computer science. In the face of Moore’s Law, it is therefore over 20 generations obsolete.

Let us hope that this little musing upon the history of the account gives at least slight pause to modern practitioners seeking to enshrine climate change as a permanent account for long range intelligence analysis. At a point in time when supercomputational problems that take longer than a few months run time simply are not run at all until the next generation of architecture advances its inevitable order of magnitude or more, it is after all more than a bit presumptuous to assume that any community entity would be able to beat or even match the kind of big iron thrown at these problems in the civilian science world. It also very much begs the question of what better use such resources might be put to for other intelligence accounts – perhaps in the classic roles that the IC has always employed supercomputing resources: cryptanalysis, automated signal processing, or even exploring the new boundaries of potential offered by quantum intelligence.

If nothing else, this bit of history has also more firmly reinforced our opinion that climate change issues are a matter best left to the academics. Perhaps once the Long War has been won – and given the timescale we believe will be needed to accomplish this monumental, generational task – then the community’s attention can turn more to the matter once again. And if the current crop of speculative forecasts prove correct, at that point in time the issue may properly fall within the window of an actionable long rang estimate.

Evidence inferred

In the rush to focus on new analytical techniques and methodologies, particularly the more complex and arbitrarily numeric variants, we think that the fundamental aspects of inferential reasoning are too often overlooked. This is by no means the stuff of such modern vogue as complicated algorithms or cutting edge research into cognitive developments. Nor is it the drumbeat of “critical thinking” applied absent real problems or even properly constructed examples.

Rather, this is the meat of what separates the analytic mind from that of the mere reporter. The problem occurs not only when building new analysts, who are too frequently shorted on the fundamentals which are vital in cultivating this distinction, but also when developing analysts’ tradecraft at the journeyman level.

Part of the reason why inference is so often given such a short shrift is that there is rarely new material of interest in circulation on the topic, when one can easily find a few hundred other resources for just about any new “hot” topic. But then again, one supposes it is hard to muster funding for a proposal described as an attempt to revisit and extent the work of a lawyer from the late 19th century.

It is for this reason we were delighted that the folks at University College London have put together an excellent cadre of interdisciplinary researchers to tackle the problems of evidence and inference, including the esteemed Professor David Schum (who first brought the phrase into common usage within the intelligence profession). The Evidence Science group is clearly seeking to forge new ground from what others have long ignored as well trod paths. Their publication set is indeed well worth reviewing, as it covers a wide range of topics central to the profession of intelligence: the persistence of discredited evidence, exploration of belief formation, and the use of rhetoric and argumentation, among many others. While only a few of the pieces focus explicitly on the intelligence domain, these exist within a selection of interdisciplinary material which also can provide its own value to the judicious reader. (It is for this reason that we can recommend the site despite the number of lawyering and quantitative analysis publications).

We think this might be just the thing for some of those in the Beltway contemplating a long weekend away from it all and looking for some not-so light reading in the wider literature.

Revisiting analytic rigour

The research currently being done at Ohio State University into the problems of intelligence analysis – including information overload, cognitive processes, and other aspects of the methodology – has from time to time caught our interest. Among the more interesting of these items now in circulation is an excellent lecture that we most highly recommend to our readers, recorded last year during the too often overlooked Google Talks series. The discussion focuses on the evaluation of analytic rigor, and means by which analysis may be strengthened.

We particularly favour the philosophy that Dr. Woods presents, which seeks to avoid dictating a single best methodology or process. We are more than willing to listen to the methodologists, but too often we find a dictatorial approach significantly at odds with the realities of line analysis. We think that the observed case study technique used in the Ohio State team’s research – something too infrequently done by many academics – is key to the validity of their findings. One cannot discuss analytic ideals without involving those who are actually involved in applying tradecraft to real problems. It is also not enough to conduct such research in artificial environments within student populations – real line analysis is too different.

We certainly cannot agree with the apparent off-hand condemnation of “folk” psychology of intelligence analysis – clearly aimed at taking on Heuer’s “bible”. While we think that there is a clear role for the methodologists and their research into strengthen analytic tradecraft, there is also a very real need for interdisciplinary adaptation from other areas of social science, as well as the kind of internal discussions that make up a key part of the maintenance of those oft-criticized, but entirely vital, guilds that are the backbone of the community.

We do however find several key concepts of great interest that deserve wider attention, including the concept of the Supervisor’s Dilemma – the balance of customer outcomes requirements and analytic resource opportunity costs against the relative depth of analytic rigour. We also find the study techniques themselves of interest, especially the concept of elicitation through critique – something we feel will likely have a far greater applicability in capturing the kind of intergenerational knowledge that the community is in danger of losing. We see the technique as one means of making more formal – and scalable - some of the kinds of subtle interactions that characterized the experiences of apprentice and journeyman analysts under the mentorship of a master.

We also find great merit in the good professor’s comments regarding the overconfidence of new analysts, and the satisificing biases that result. We definitely have observed a level of arrogance in too many new hires – and especially those coming out of the intelligence studies programs. The first lesson that an analyst student should learn is the fear of God – and of their own error. Too many programs of instruction are not affording the student the chance to learn that fear from the visceral experiences of their own mistakes, and to take away from the experience a humility that will cause them to productively question their future work toward its improvement. Such experiences are far better gained when the consequences are not fatal, in line with the lessons taught by Red Flag.

There is much food for thought in this lecture, as well as the contributions to the literature that the Ohio State program has generated. We will no doubt have further commentary on the subject in the near future.

Revisiting Twelve Angry Men and legalism in intelligence analysis

For a number of years, the classic black and white film Twelve Angry Men has been a frequent teaching aid in introductory analysis courses dealing with the basics of evidence and argumentation. The conventional use of the film is to provide an accessible means of deconstructing a fictionalized scenario for students with little prior experience with formal debate. Given the current decline in public education, this helps remedy a basic skills deficit that is unfortunately and increasingly all too common.

However, it is with interest that we observe the controversy that has erupted once again over this fifty-one year old movie. The criticisms that have been leveled against the underlying premise of the film deserve some additional consideration – not the least of which because they point out the serious problems in applying much of what is taught as legal logic to the unique problems of the intelligence domain.

The Spectator’s argument surfaces one of the reactions common among many students, but in a far more articulate fashion than any entry level professional might be expected to voice. In essence, this criticism is based on the need to focus on the external worldview, rather than the tactical maneuvering in the courtroom that such kinds of arguments inevitably devolve towards. It is a quite valid point, and among the reasons that we have long decried the trends towards creeping legalism that have lately come to dominate intelligence work. The bulk of a lawyer’s litigative activities – and therefore a disproportionate degree of their education and professional experiences – are dictated by entirely tactical considerations that apply nowhere else but within the limited framework of the legal system. Too often this is easily forgotten, to the detriment of the strategic picture – and the accuracy and veracity of analysis. We have written on these problems before, but to be frank we had rarely considered the myriad of ways in which – by borrowing from the older legal profession’s traditions in teaching basic logic and rhetoric – the intelligence academia may continue to contribute to these unhelpful trends of cognitive bias. Among these, of course, are the kinds of ludic fallacy identified by Nassim Nicholas Taleb.

The folks at Volokh Conspiracy take a different tack, arguing for greater consideration of interdependencies within factors under examination in the film’s fictional trial. This is also a very useful approach for discussions with students, many who likewise tend to view evidence in isolation. It is particularly appropriate when covering structured analysis techniques such as ACH – and one that rarely touched upon, if only due to the frequently too shallow examples offered to illustrate the methodology, which are unable to support a more robust discussion.

Westminster Wisdom rises in defense of the film, and illustrates the more important but also often overlooked value to the piece in the intelligence studies classroom – the discussion of uncertainty. The intelligence professional will always work within a framework of ambiguity, doubt, and frequently, deception. However, the role of intelligence is not merely to reach a lower standard of proof than that used in a criminal trial (or even the lesser civil threshold), as is commonly taught (and in particular, a tenant of faith within law enforcement intelligence). Rather, intelligence’s purpose is to provide accurate insights despite such uncertainty; and where absolute accuracy is not possible, to bound the space of uncertainties for the policymaker in a manner that supports informed decision-making.

In light of all of the foregoing, we continue to search for good alternatives to the film for use in the classroom. Our search is also driven by the simple fact that to the Millennial generation, the black and white format is very nearly entirely alien. It creates such a visceral negative reaction that the first ten to twenty minutes of the film are simply an orientation to the unfamiliar environment. The pace of the thing is also glacial by modern standards, and particularly so to minds attuned to rapid multi-tasking and immersive information environments. While one can make all the arguments one likes about the need for sustained single focus attention, their native preferences are indeed more suited for the kind of world in which they operate as intelligence professionals than the Industrial era conventions that black and white film represents.

The single set format, and the emphasis on argumentation, has however made it exceptionally difficult to find a substitute. Our best – but admittedly still imperfect – alternative has lately been the 1999 film Deterrence, which offered a President’s decision-making process in a nuclear crisis while snowed in at a small diner. Unfortunately, the film’s scenario is constructed around a fictionalized Iraqi dictatorship – which in the modern politicized climate often steers debate too far astray of the real purpose and into the debate over Operation Iraqi Freedom. It also pre-supposes a certain level of student knowledge regarding nuclear warfighting and mutually assured destruction strategy: something not always guaranteed in the post-Cold War cohort. This sometimes makes for quite interesting discussions, to say the least. These are the same problems that incidentally also led us to abandon using the old JMITC exercise that relied on the film version of Hunt for Red October as a notional scenario from which the students would develop practice analytical pieces. While, as a friend recently reminded us, that particular film ages very well as such things go, we recognize that writing intelligence on Soviet era ballistic missile submarines is an anachronism to which few students will respond well –and one that does not serve their real and current professional needs.

All in all, teaching intelligence with films – for as engaging as the technique might be for students bored with lecture – remains a difficult proposition. We hope that in time the development of newer tools for digital animation – and the kinds of interactive scenarios that new gaming engines permit – will render the question entirely moot. But the cultivation of young professional minds remains a terribly stubborn business, and one that is not frequently improved by new technologies. We do wish to see such improvements become effective, and preferably in the near future. But given that the same promises have been made since around the time black and white films first graced the big screen, we remain skeptical.


h/t Overlawyered

Delving deeper into prediction markets

Michael Abramowicz of George Washington University has been guest blogging at Volokh Conspiracy for a short time now, and he has given us much food for thought on the topic of prediction markets. There is easily enough material for an entire book, and unsurprisingly, he has written one (that is now on our must read stack); as well as his own blog site. The Volokh post series has been:

• An Intro to Prediction Markets and the Liquidity Problem

• Prediction Markets vs. Conventional Wisdom

• A Quick-and-Dirty Empirical Defense of Prediction Markets

• Manipulation of Prediction Markets

• Deliberating with Prediction Markets

• Predicting Decisions and Their Effects

• Normative Prediction Markets

• Why Normative Markets?

• Nineteenth-century prediction markets

We will no doubt have more to say on the topic ourselves in good order. However, it is a subject that deserves deeper reflection, especially given our acknowledged skepticism of such efforts – and our general distaste for attempts seeking to create artificial numeric precision.

Nonetheless, we are quite grateful to the author for his work, which offers a unique contribution to the literature in an area of great interest to the IC. Whatever one may think of the technique, it is worth exploring with the same rigour as any new methodology.

The perils of arbitrary and false precision

We find quite unhelpful the recent academic obsessions over estimative language – largely an exercise in the introduction of a numerical system which offers a degree of apparently comforting but entirely arbitrary, and therefore utterly false, precision. It seems however that we are in one of those cycles which seem to come along in the intelligence community every few decades or so, in which the numerologists and other soothsayers attempt to reshape the profession into their own desires for a more “scientific” practice.

Let us be clear. There are times when quantitative analytic methodology is vital – but there are far more situations in which it is misapplied, misunderstood, and entirely out of place. The latter comprise the vast majority of scenarios in which analytic tradecraft is called upon – not the least of which may be attributed to the highly unbounded and indeterminate nature of the problems with which we must grapple. And any time in which a quantitative basis has not been established, the insertion of numerical percentages for predictive purposes is little more than a farcical exercise in arbitrary selection. Over time, you may attune a group sufficiently in order to calibrate its judgment of these percentages in such a way as to create a consistency within that shared hallucination. However, this does not alter the underlying fallacy upon which such a house of cards is built. This is clearly shown in the number of cases in which the naive predictor is a better estimate of potential than the much vaunted group of experts’ judgment. Thus even in finance, the most precise of arenas, built upon the foundation of values, you will find predictions expressed equally alongside hedges – and the market littered with those who have failed to impose arbitrary figures on a highly indeterminate problem.

One of the greatest challenges in intelligence analysis is to understand the limits of prediction when going about the hard business of estimation. That understanding should shape the analyst’s focus on what ought to be examined for predictive possibility. These are, properly: the scope and nature of trends, drivers, and future scenario outcomes – and not the capricious shadings of difference between mathematical expressions of probability.

Estimative language has not been expressed through probability percentages for the sixty plus years of the intelligence community’s modern incarnation for good and well contemplated reasons. While the abstraction of the clean and sterile realm of mathematics is often a welcome change from the messy and hard realities of intelligence, that abstraction too frequently is used as a shield and an intellectual refuge for those unable or unwilling to embrace the challenge of actually doing intel.

Scientisim in intelligence analysis is a particularly seductive heresy. It offers the false promise of greater insight, should only additional efforts be applied more systematically, more rigorously, or with more and better data. But it has not been given unto us to see the future – no matter how carefully we might craft our equations. We may simply chart the boundaries of its outlines, and discuss the implications within the uncertainty space so described.

We have no doubt that we will revisit this discussion in short order. For now, however, we would close with an excellent reminder of the vast gulf of differences that may be concealed with that change of a single degree of significance in numerical expression. Originally produced for IBM, this admittedly dated video still serves to explain the staggering concepts of scale in a world of large numbers. (h/t to Thoughts Illustrated for pointing out its online incarnation.)

A hard look at prediction markets

Rarely does an analytical methodology garner attention in the manner that has marked the discussion of prediction markets. From controversial origins to increasingly widespread public adoption, we think more pixels have been spilled on this single approach than on perhaps any other methodology short of ACH.

We have written about such techniques before. Perhaps we might group them more generally under the moniker of arbitrary quantitative forecasting. Arbitrary, for the numbers themselves however derived have only relative meaning in the assessment of probabilities (including even financial data, which although it carries with it information about the state of a transaction series or commodity, responds as much to the complexities of interactions between financial entities as it does to those factors of relevance for intelligence forecasting.)

It has been difficult, however, to evaluate the effectiveness of the technique amidst all of the hype. Certainly, we know of no significant influence on ordinary analytic tradecraft. The real business of intelligence continues much as it always has. This does not necessarily invalidate a methodological experiment, for there is certain room for more specialized vehicles to address unique problems or support new product lines. This is the usual fate of a new and uncertain methodology, and is not a bad thing in and of itself. (Although we do not that adoption of new methodologies has been recently accelerated, which we can attribute in part at least to the more widespread discussion within a growing literature. A new technique or approach might have lingered for decades before seeing significant use, but now may find a home – even if in a specialized shop – within months or years.)

Validation has always been the bane of methodologists. However elegant their theories, they are doomed to academic irrelevance unless adoption occurs across a sufficiently representative section of the community. And absent validation, adoption – especially in cases where significant implementation effort is required - will always chancy. In the face of production pressures and surge requirements, analysts will in almost every case fall back upon processes with which they are familiar – structured or otherwise. Prediction markets by their very nature tend to require a substantial up-front effort for highly uncertain results.

We are thus grateful to the folks at Google, along with coauthors from NBER and Dartmouth, for publishing some of the first real results of their internal prediction market. The study covers nearly three years of the operation of an exchange which handled over 70,000 transactions – each conveying a degree of belief on one of almost 300 particular questions, on behalf of 1500 active employees (although nearly 6500 held accounts that were not used.) Interesting, they identify unexpected influences due to physical proximity, as well as the impact of cognitive bias towards optimism based on employee fiscal considerations created by Google’s rising share price. Also quite interesting was their observation that new employees were more influenced by this bias, and that staff with longer tenure within the firm tended towards more calibrated judgment – a not inconsistent phenomenon within any analytic activity.

As warrant to the authors’ point regarding proximate location influences on information sharing, it was also revealed that Google employees moved offices approximately every 90 days. If ever there was a indicator of a complex and unstable system… but of course, we are aware of quite a few community elements that would meet or even exceed this frequency.

At least a third of all market questions were purely “fun” topics, while nearly half did not have direct impact to Google. This begs the question of how much of the activity was merely socialized gambling using virtual currency vice the exercise of deliberate judgment regarding the potential future environment – something that will plague almost any prediction market collaboration. While fun helps drive adoption, and play can lead to divergent insights, it is easy to envision such a mechanism as becoming a drain on the hard questions of the real topics.

All in all, the paper is well worth reading and carries with it quite a bit of food for thought to sustain those debating the utility and applications of prediction markets within the intelligence community. We admit to a growing skepticism regarding the value of the methodology that this study has only served to reinforce. Given the total time, resources, and intellectual energies required to support such an endeavor, these kinds of outcome do not in our view necessarily justify the effort. However, we remain open to the potential that such mechanisms capture effort which might otherwise be entirely undirected, and therefore may create insight where other techniques would not. These remain in our minds open questions, and worthy of further research.


h/t Marginal Revolution and Midas Oracle

Arguments in intelligence history

We occasionally note the scrum which emerges as the topic of intelligence history is approached within the context of larger political science debates. In this case, Sic Semper Tyrannis has resurfaced the debate over the utility of social science derived approaches to intelligence analysis practice; centered around Sherman Kent himself and the original Ivy League influenced R&A, first raised back in 2005. The current piece expands the discussion into a wider examination of the use of intelligence within a supposed political school of thought allegedly shared by several recently prominent decision-makers. Though we find the formulation “neo-con” to be a particularly unhelpful analytical framework, both too heavily influenced by recent political rhetoric and prone to use as a shorthand masking too many anchored and unexamined assumptions; the piece does offer some value in summarizing a set of arguments of the contemporaneous back and forth of Soviet analysis that is rarely well captured in modern study. Also quite interestingly, whatever one’s take on the issues themselves, the discussion does illustrate the impact of the academic and privatized intelligence tradition on key community history, influenced which are too often overlooked and thus all too rarely taught.

We also note the very differing perspectives on the first exercise of alternative analysis, the famous Team B studies, to other views about the experiment referenced by Former Spook over at In From the Cold. It’s an old stalking horse of the intelligence academia, and in our opinion, too much of the subsequent debate over these cases has been influenced by contemporaneous political maneuvering intended to protect a government monopoly on intelligence in the face of a unique privatization challenge. We note that alternative analysis has now been enshrined into community practice – thus the lessons of the original competitive analysis exercise could not have been nearly so negative as many subsequent commentators would insist, whatever one might believe about the relative estimative accuracy of each team’s conclusions. Tradecraft may well be advanced by even imperfect experimentation, so long as methodologies are refined in subsequent iteration.

We also wish to thank Non Partisan Pundit for the kind words in the comment thread, and hope we may continue to strive to earn the appellation “not only well written and informative, but it's also refreshingly free of politics.” An intelligence practitioner is always intended to be professionally apolitical – a fact that we believe is not stressed nearly enough to the younger generation in an environment of highly politicized media, and the ongoing football between the Hill and the Administration (and back again). We see not only the ethical obligations inherent in this requirement, but also the practical benefits to the apolitical approach, as exemplified in Tetlock’s research into expert judgment, best approached through his recent work cast in Berlin’s metaphor of the fox and the hedgehog. We also look forward to the Pundit’s further examination of intelligence studies issues, such as the discussion of the effect of “yes men” within closed regimes.

Differential problem solving

Business and competitive intelligence practitioners are one of those often overlooked sources of insight into analytical tradecraft in situations, and with individuals, very different from those typically encountered on the normal pol sci / national security track. Many of those insights have surprising cross domain relevance to the intelligence community proper.

Thus it is with the following item examining the behaviors of various (non-intelligence) technical minded types when confronted with an analytical problem. The differences in mind-sets, approaches, and even processes employed by the two groups make for a fascinating little vignette regarding comparative styles of critical thinking. (While we may object to the conclusions drawn in many other items on this particular blog, we do occasionally find an item or two of interest. And of course, given that the author resides in Kent in the UK, we could not help but notice the blog.)

We recall a similar little thought experiment that was routinely conducted during a strategic intelligence analytic tradecraft class once commonly offered around the community. The exercise involved a group of practitioners and managers, usually from very different organizations and entities, trying to tackle a new “emerging” target account from soup to nuts – requirements definition, collection management, and analysis and production planning. We likewise recall the very differing approaches and styles that surfaced during those discussions. At the time, common wisdom tended to chalk this up to institutional culture – and in particularly, the reinforcing power of organizational stereotypes (and expectations of those stereotypes). A more subtle interpretation would be to examine the extent to which individuals came to those organizations, and to succeed within them, that were more likely to embrace those cultural norms and even stereotypical responses.

The benefits of this kind of cross domain discussion (if you can keep the participants from killing each other) is perhaps one of the strongest arguments we have heard voiced regarding the need to break the old ivory tower model, and to integrate collectors, analysts, and operators throughout key accounts – in close proximity to the policymakers.

Epidemiological intelligence reconsidered

Some time ago, persistent virtual worlds made news for an unusual incident in which a plague spread widely through a massively multi-player fantasy game in an unanticipated fashion, due to the complexity of the system’s design. There was a great deal of speculation as the ramification of this event – some of it mirroring the longstanding discussions around self-replicating “gray goo” and other nanotechnological questions, some of it quite unique in its own right. The event even spawned serious scientific papers, and responses by other researchers.

The question of the potential utility of virtual worlds for examining epidemiological effects in simulation remains a fascinating area of study. It is a natural extension of other research attempting to track in a similar fashion the effects of viral ideas – memes – within simulated virtual populations for information operations / psychological operations studies. This is the sort of analysis which may dramatically alter the manner in which medical intelligence professionals approach their craft in the future. The watch desks of tomorrow, rather than being tied to a series of open source intelligence portals and medical information database feeds, might well also be linked to a shared situational awareness simulation, with the ability to rapidly generate scenario projections based on new reporting or analytical inference. Certainly, there has been enough cross-boundary interest in the problem that we expect to see surprising innovation in the near future. It is certainly long overdue given the need, especially in the face of recent exercises attempting to examine the impact of pandemic scenarios which were certainly less than robustly designed and executed.

In its own way, this is by no means a new problem. Among the most interesting books we ever had the pleasure to peruse on related subjects was Plagues, Poisons And Potions: Plague Spreading Conspiracies in the Western Alps c.1530-1640. We discovered this work once upon a time buried in the back shelves of the bookstore of the British Museum, and discovered it had a striking relevance for those interested in non-state actors’ motivations in biological terrorism / biological warfare incidents. The book is based largely on primary source records from Swiss city governments throughout the Savoy which were suffering through major disease outbreaks. A recurring series of cases are documented in which individuals deliberately attempted to spread disease to uninfected populations – motivated by cult conspiracy, simple hatred, and criminal profit. Given the evidence of other recent cases involving deliberate biological infection, it appears human nature remains little changed in nearly five hundred years.

We also note in a related vein the Swedes’ take on the potential dual use implications of some medical intelligence programs. While we differ with their analysis in that we are confident in the benign nature of US and allied programs, we would definitely see reason for concern in investments in such activities by some states (or their non-governmental organization counterparts) with proliferation interests.

Epidemiological intelligence remains a fascinating discipline in which the contributions of a number of different professions converge in a manner that is very rare in the community. There remains much ground for formal study to advance both the analytic tradecraft and the literature of the discipline, and perhaps to inspire similar interdisciplinary approaches in other areas of the profession.

The forgotten art of descriptive analysis

Descriptive intelligence analysis is not typically considered real analysis by many academics, who usually represent the work as the bottom rung of a pyramid in which their attention remains focused on the supposed “higher echelon” tasks. These same academics tend to ignore current intelligence of all sorts, and all too often also intelligence activities conducted at the tactical level – their interest being limited largely by their lack of understanding of any products other than the most famous national and strategic FINTEL such as the PDB or NIEs.

But stop and think for a moment about how many times one encounters in the practice of the profession complex, multifaceted situations with literally decades worth of reporting that must be summarized – and in a few pages or a few minutes' briefing, convey the essence of the thing in order to even begin to understand events that transpire in the world. This is by no means an easy task, and typically may be as difficult as the kind of push for predictive intel that we see for the larger and better understood issues. That descriptive analysis seems mundane in much of recent literature is very much a product of those who work from unconscious bias and stereotypes of situations, without real depth on an account. One could even say that most of the real intelligence challenges in Iraq, since the start of OIF to the current Awakening, have been in first understanding and describing complex dynamics and personalities in order to more positively leverage the multinational capabilities to good effect. (There is a reason efforts like the Human Terrain System have been created in order to aid in better description of complex targets within the cultural environment.)

The lack of respect accorded to some of the most common and critical tasks within the community has also led to the dismissal of a whole range of environments and tradecraft which have grown more common even as they are less discussed. The proliferation of new watch centers and the growth of the fusion center concept has created new demands for real time, 24/7 intelligence support – often relying on new sources of information not previously considered by the intelligence community. After all, the needs of homeland security are far different than foreign intelligence in many cases, particularly when one examines all hazard issues and matters of greater state / local / tribal concern than the traditional CT focus. There has been simply no real effort within the intelligence studies academia to understand these environments, and to examine the changes in tradecraft wrought by new collaborative technologies, organizational structures, and functional processes that have come to be considered best practice in that world. Despite the dozens of centers, and the thousands of analysts, operators, and watchstanders employed in this segment of the community, it remains virtually unknown in the literature of the profession – not least of which because it would involve examining issues perceived to rank at the low end of some notional representation culled from decades past. And it is not just a homeland security thing - there has been an equally important growth in joint intelligence operations centers and fusion task forces within the national intelligence side of the house.

We sincerely hope that some bright young thing in one of the new academic programs will soon take an interest, and focus some much needed attention on this area.

Pondering anonymous wiki usage

We note with great interest the research findings of the Dartmouth team that examined the role of anonymous contributions to Wikipedia. This work confirms what many have long suspected – the role of many individual experts contributing in a small area can be as vital as the long term “gardening” and other high commitment roles of high frequency wiki users.

This tracks interestingly with an alternative analysis first provided by Aaron Schwartz on the true distribution of authorship in Wikipedia. His work challenges conventional wisdom that only a few hundred individuals have been responsible for the majority of the production, showing that the contributions of these otherwise limited participants that actually provide the bulk of new content. The high participation individuals provided most of the structure, formatting, and debate.

None of this is terribly surprising when one considers the dynamics of expertise and contribution in other endeavors. But it has profound implications for those seeking to use the dynamics of participatory production models to create things of enduring value within the intelligence community. There is an inherent distrust of anonymity in the IC, and in a professional environment one’s reputation is not just at stake for a hobby but for the weight of one’s “real” work. How much has the intelligence community denied itself potential contributions of value (and reliability) through some of the choices made regarding anonymity in its wikis (or blogs)?

We come down strongly on the side of appropriate veils for the online environment, of course. Not that we wish to be the man behind the curtain (although professionally, some of us not in the more active side of the house are always more comfortable on the dark side of the one way glass), but rather so that ideas stand alone and can be discussed independently of the agencies and cultures which produce them. We have been accused of ill will on more than one occasion for our anonymity and group voice, but it is simply a desire to extend the debate on professionalization free of the conflicts of personality and organization. (It is also a function of the unique terms under which we are able to continue this venture, but in this it is a happy convergence with our intended outcomes for the blog.)

The parallels of Wikipedia assume however that the IC intends to create an encyclopedic work of its own. It is far from clear that this is what Intellipedia will be, let alone any of the other smaller and more focused wiki production environments. There are several other distinct roles evolving for wikis as the technology is bent to new situated uses within small groups – from watchstanding to warning, from dynamic production processes to shelfware reference replacement. The experimentation is really only just beginning – and for this reason, further real research is needed from the intelligence studies academia on both the open and dark side versions of these tools.

No maps – but perhaps guides – for these territories

One of the under-examined aspects in the adaptation of new web n.0 technologies to the intelligence community has been the potential utility of social bookmarking (in the model of del.icio.us and other related sites.) This has been in part remedied in the most recent Journal of Intelligence and Counterintelligence, in an interesting article on the “The Application of Social Bookmarking Technology to the National Intelligence Domain”. The authors come out of the Monterey Terrorism Research and Education Program, a fascinating group that has done good work assisting characterization of the trends in global terrorism from an outsider and OSINT perspective. (We continue to be fascinated at what role these sorts of projects have in exploring and validating analytical tradecraft in new contexts, and against new problem sets. We can think of no better example of this within the community itself than NCTC/Worldwide Incidents of Terrorism.)

What is interesting is that the concept of shared pointers to interesting and useful information sources is by no means a new thing within the intelligence community. Methods of such sharing – from the humble file card to the group email list – have had a long and successful history. Most, however, occur below the surface of recognition under the general rubric of mentorship. But it is clear who has such connectivity and sharing, and who doesn’t, when one compares analysts’ knowledge and situational awareness. It is not clear that a social bookmarking technology is the real answer – although the tool may help ease the formation of the kinds of emergent behaviors that drive the real successes of this sort of collaboration – the highly connected interdisciplinary individuals who routinely cross multiple cultural and organizational boundaries.

It is those rare individuals who essentially act as guides within the unexplored territories of information sharing. They do so, often at great personal risk, as the volume of material they typically pass between large numbers of individuals – often on the order of dozens of items a day to hundreds of folks in informal networks – can result in numerous administrative or political problems given even the smallest of mistakes by any individual in the chain. Despite this, they are vital to the work of the community. And while some hold official positions as liaison officers, most simply happen to enjoy an unusual mixture of independence, immunity from pressure, and a very large personal social network built over time through friend-of-a friend referrals. And in most cases, their organizations do not even understand the value these individuals bring to the table – sometimes even far in excess of anything else that particular shop might be doing.

Encouragement of these guides – be they mentors, gardeners or the near autistic savant – is more than technology, and demands as much focus as the tools and toys. Guided information consumption is critical to the cultivation of the analyst, both for its informative and situational awareness value but also for its inspirational effects. These are hard to measure, and difficult therefore to justify investment in. Nonetheless, without them, there would be far fewer exceptional intelligence practitioners today; and likewise those gaps will grow in tomorrow’s workforce if there is insufficient focus on such efforts – formally or informally.

Counting the minutes

Those with even a passing interest in counterproliferation issues will no doubt be well aware of the Bulletin of the Atomic Scientist’s infamous Doomsday Clock. Tao Security examines the implications of using a clock to convey threat information, and in particular the implications of shifting what was originally designed as a publication art into a watch and warning mechanism.

Many of these lessons, we think, could be taken to heart by those seeking to implement an ever wider array of threat level displays for almost every new issue that arises in a multi-agency or cross-account context. Too many of these efforts seek to differentiate themselves by creating their own standards, hoping to be cited with the same frequency as the more recognized measures of warning.

However, all face the same questions of setting an initial baseline, and of calibrating the changing conditions to the consensus of expert judgment based on imperfect and often contradictory indicators. These by themselves are not intractable problems – in fact, they are quite familiar to those routinely engaged in the business of analysis. (Although many now pushing for new threat level measurements do not themselves come from the analytic tradition.)

What complicates these new efforts is the effective communication of these measurements – often done in color coded fashion (to mimic the DHS National Threat Advisory), or through some combination of alpha-numeric or word designators. These are displayed variously as bars, speedometers, and a variety of other graphical tricks pulled from simple spreadsheet charts so favoured by accountants. And we can think of few less productive ways to spend the valuable time of intelligence professionals than group debates over the exact shade of color, or font, of one of these new graphics.

Effective warning in intelligence is a difficult task that is as individual as the consumers that it serves. We think the clock model, and its counterparts of the speedometer and bar graph, deserves to be left to rest. In much the same way as the intelligence community has moved away from numeric designators for source reliability and accuracy and towards more descriptive paragraph summary formats, it is long past time to reconsider how warning conditions are conveyed to decision-makers in a more robust fashion which clearly brings across not only the underlying indicators but also the key changes which drive current judgments.

Stratfor blogging

George Friedman, Chief Intelligence Officer for the private intelligence firm Stratfor, is now blogging. We view this development, like much of Stratfor’s work, with mixed sentiments. On the one hand, Stratfor as an entity has done much to legitimize to concept of the privatization of intelligence – not merely in terms of government contracting, but also in terms of serving clients outside of the traditional community’s consumer list. Many multinational firms have much the same needs for robust geopolitical FINTEL as any government department, and Stratfor helps fill that niche.

Stratfor also one of the few private sector intelligence shops that has been in business long enough that it might perhaps be a profitable enterprise (although of this we are uncertain, as we have no inside information regarding its financial status). There are only a few other shops that may even attempt such a claim, however, many are affiliated with larger institutions – such as Oxford Analytica, or the Economist Intelligence Unit. The other major private sector intel concerns - the travel intelligence shops such as Control Risks, iJet, and International SOS - have faced quite varying balance sheets over the years, and much of their revenue comes from services outside of the core intelligence lines – K&R, due diligence, corporate overseas medical support, or continuous injection of new investor’s capital. In previous years (before their acquisition by Verisign) we might also have included the old iDefense shop in the list of profitable private sector intelligence efforts, but rumours of the shop’s troubles in its newest incarnation give us pause – including key indicators such as their recent loss of a number of key people, the complete loss of its branding into its parent corporate identity, and now the potential sale of the unit as part of Verisign’s divesture of its managed security services portfolio.

Stratfor thus stands somewhat apart from the rest, as an independent shop in continuous operation for over a decade. But in that decade, its track record has been exceptionally unsteady. It first made its bones during the Kosovo crisis, with unique new information sources (in an area where few shops had anything at all) and the occasional innovative but solid analytic line. Its attempt to act as a “global” shop in the mould – and even, boastfully, claiming competition with – CIA, did not fare so well over subsequent years. Occasionally, they have a good piece. But often their analysis reflects their hiring strategies, which Friedman himself proudly holds up as an ideal model – the selection of young students, fresh from university, with no prior intelligence experience. Stratfor claims this allows them to build new analysts with no “bad habits” that might have been learned in the intelligence community. However, it ensures that they have a workforce that will always lack substantive experience, creating a shallow bench on accounts. This can be quickly and professionally fatal on hard targets, or when they step into areas in which existing analysis is a career long affair for an entire analytical sub-specialty (such as oil market dynamics). While we are great believers in the value of the beginner’s mind, and of the importance of Smoking Mirror, we thinkStratfor’s approach goes a bit too far.

Their analysts also reportedly face a brutal gauntlet of forced predictions for quarterly and annual forecasts on their accounts, which are tracked and if found to be repeatedly inaccurate over time, results in their dismissal. While in theory an exercise in accountability, one of the enduring lessons one should learn when managing analysts is put forth by Nassim Nicholas Taleb. In any given population of forecasters – NNT chooses Wall Street traders as an example, due to his experiences and interests – there are a certain percentage that will be accurate each year simply due to pure chance. Year over year, this will result in a small number of “successful” forecasters, who at the end of any given evaluation period will appear disproportionately better than their peers. Again, while we are strong believers in analytic accountability, and continuous improvement in analytic tradecraft and accuracy, we are acutely aware of the limitations of intelligence as an activity, and the impact of random chance.

We will be interested to observe the development of Stratfor’s enterprise blogging over time. We would be most glad also to see its line analysts surface from behind the curtain, and have their occasional chance in the limelight that is typically dominated by a few senior faces. And we hope to see more than just a reposting of its public products – though it is an interesting experiment in feedback on those products, and one that takes no small amount of courage (for which they should be rightfully commended).

But in the blogsphere, it is not enough to rest upon one’s reputation, and to treat shallowly across wide fields of inquiry. There are real subject matter experts that may emerge unexpectedly in this medium, and these can pose both a challenge to the unwary or arrogant, but a tremendous opportunity to those willing to change and adapt to the new environment. For example, their latest piece on Blackwater could well have been far better informed with the cooperation of established bloggers that have long discussed the impact of PMCs (first in our minds is of course Mountainrunner, but also the boys at Coming Anarchy, and Shloky). Let us hope that Stratfor learns from their example, and involves such SME’s in their future products.

All in all, we wish Stratfor the best in their new approach into the blogsphere. It is rare to see such a conversation begin with a private sector intel shop, and we would see it continue. Of course, we hope they will discuss their tradecraft and forecasting methodologies in more detail – perhaps even proving our impressions wrong. But either way, we hope they will further contribute to the literature of intelligence from the basis of their unique experiences.

New applications for patent analysis in competitive intelligence

The exploitation of patent databases for competitive intelligence has been a key staple of much of the private sector’s intelligence work. This has been particularly a feature in consultancies with strong scientific and technical accounts, as well as the major manufacturing and R&D houses in the pharmaceutical and materials engineering sectors.

The Economist profiles a new method of mining patent information as feedstock for genetic algorithms designed to rapidly develop problem solving approaches for those wishing to find a novel alternative free of patent encumbrance (and in the process, avoid legal difficulties that may be created by the re-use of existing solutions).

The wider adoption of this technique would likely drive additional interest in what has been a longstanding element of competitive intelligence tradecraft. This follows an increasing trend within the private sector’s evolving intelligence sophistication – the reliance of many firms on intelligence products and services not merely for the executive level (as had long been the insistence of those wishing to assure C- level buy in, or model their own efforts on common IC publications processes such as the PDB). Rather, many of the successful new applications of intelligence are tightly integrated at the working level in the corporation’s various business units, and directly support the line responsibilities of teams of industry experts through specialist contributions.

To a large extent, this shift in private sector CI also mirrors the evolution of many intelligence efforts in other environments, which trend now towards increasing numbers of analysts embedded with units of operations professionals, in a distinct break from the supposed ideal of the “ivory tower” analyst organization divided functionally and geographically from the consumers they are supposed to support.

We might add that the same genetic patent analysis approaches, may also certainly have applications beyond the private sector. One can easily envision counterproliferation purposes for the same genetic algorithm approaches, particularly given the starting point of known weapons systems designs and the existing stable of scientific expertise that may be present in a given proliferant state’s program. The genetic algorithm might suggest alternative materials or processes that an adversary may attempt to substitute for prohibited components in a military or dual use acquisition program.

h/t Slashdot

Understanding the evidence of events

A few items have continued to trickle across the nets which bring the all too often forgotten works of AFRICOM to our attention.

The first of these comes from the ever perceptive folks at Coming Anarchy, who note the continuing engagements involving host nation and US forces tracking irregular forces in the wilds of the Sahel. The Taureg rebellions in Mali and Niger are in one sense the very real stuff of which 19th century adventure novels were written, yet at the same time they are the concrete manifestation of the Global Guerrilla / 4GW+ problem set in a small, poor, and out of the way place in the world. A place that just happens to border key areas with other known problems also involving restive populations heavily influenced by militant Islamist ideology….

More troublingly, though, we also note the Economist’s puzzling reaction to the body of public press reporting regarding Islamist – and in particular, Al Qaeda, activities in the region. They editorialize in an article on the dynamics of the rebellions: “But apart from the odd smuggling deal over guns, drugs or cigarettes, no solid links… have been established”. This despite continued open source information regarding the continued expansion (or retreat, depending on one’s perspective) southward of the Salafist Group for Preaching and Combat, who have re-branded themselves as the Al-Qaeda Organization in the Islamic Maghreb. (And while we may not always agree with their analysis or policy recommendations, International Crisis Group has published to its usual standard of excellent research regarding these issues, as has the inimitable Douglas Farah – perhaps the first journalist to seriously cover the lesser known transnational aspects of global terrorism networks operating in Africa.)

We find a persistent form of cognitive bias in this sort of thinking as demonstrated by the Economist, which would deny the evidence of events. It is one that has increasingly come to dominate policy and intelligence related discussions in many institutions. Let us call it the dissociative fallacy.

The dissociative fallacy is the tendency to ignore linkages between issues based on pre-concieved ideas regarding how such linkages manifest - ideas typically based on unarticulated assumptions, and frequently informed by popular stereotypes and other fictional information. It rejects the actual form of associations when they do not meet an idealized a priori postulate.

The dissociative fallacy is a natural development for those that perceive their role to be primarily centered around minimizing disruption and managing crisis, particularly as those acting in such roles often encounter deeply entangled issues which are not amenable to solutions in part. It is understandable, if not acceptable, that they should then seek – consciously or unconsciously – to reduce the number of factors at play in discussions regarding the issue. But it is a dangerous way of thinking that just as surely leads to intelligence failure and intelligence surprise as any other form of cognitive bias.

We see this fallacy at play in counterproliferation discussions, used to explain away any given set of dual use or even prohibited materials as unrelated to a weapons program end use. The thinking goes that unless it the material under examination is part of a “stockpile” – with attending implications of bubbling vats and ranks of missiles arrayed in Soviet era precision – that it shouldn’t count. (Something all too common in the post OIF era, as the “lessons learned” of the pre-war estimates continue to be applied before the final judgment of history – and the full accounting of the regime’s activities during the 2002-2003 time period – have been reckoned.)

We see it frequently cited in counterterrorism cases, in which travel linkages, communications exchanges, ideological references, and self-identification are discounted in favour of theories of self-radicalization and explainable mental disorders. And for as much of a threat self-radicalization does actually pose to permissive democratic civil societies, all too frequently in recent experience substantive associations to more organized terrorist entities have emerged during post-incident investigations of supposed “lone wolf” actors. But since most of these relationships do not fit the Hollywood-driven assumptions of how terrorist associations are formed, they are unremembered until after the next attacks.

The fallacy itself is perhaps most prevalent in other transnational issues area, where single case study examples of an incident type are merged with implicit assumptions regarding organized crime or other threat activities, creating a “standard” model which does not begin to encompass the range – and frequently, the strangeness – of real world cases.

For as commonly as it appears, there is very little in the literature on this point. There is a clear need for academic research in this area. Perhaps some bright young thing from one of the better institutions might taken an interest, and grace us with unique study and innovative insight into the nature of the problem.