Financial crisis and changing paradigms of warning intelligence

The continually interesting competitive intelligence forum at Ning has surfaced a discussion which has been much on the minds of a variety of intelligence professionals in both the government and private sector given the cascading collapse of a number of major financial institutions: Was this financial crisis a warning failure? And if so, the natural corollary inquires into the cause and origin of the failure.

In our view, these recent events very much represent intelligence surprise. If nothing else, the unexamined higher order effects of complex financial relationships involving vast sums of cross-border capital flows is far outside the traditional realm of political and economic intelligence, at least as it is usually practiced in the government world. And the rapid contagion dynamics within the financial markets prove that the events are likewise beyond the traditional scope of competitive intelligence, where it is rare that analysis takes into account such sweeping changes across the landscape and its players. Whether this surprise truly rates elevation as a Black Swan, as some commentators have suggested, is itself also open to debate.

There is ample evidence that early indicators were visible, and even that many commentators had previously weighed in on the mounting risks and dangerous uncertainties inherent to the increasingly complex layers of traded instruments, derivatives, and debt that lurk at the center of the current crisis. However, warning is a process – not an event. It matters little that in hindsight one can call out the prescient among the punditry and politicians, and cast blame on those that assumed business would continue as normal against the backdrop of ever increasing housing prices. If warning did not reach, or impact, the right decision-makers – as there is mounting evidence that it clearly did not – then the process of warning failed.

But let us examine this more closely for a moment. Who exactly were the right decision-makers in this crisis? The primary lending institutions? The traders and market makers that were the primary players in moving these instruments? The investors, fund managers, and sovereign wealth entities which funneled so much capital into fundamentally unstable market positions? The risk managers at any of these firms, responsible for anticipating the potential downside of complex financial positions? The world’s various central bankers? The regulatory bodies or their political masters in the parliamentary and executive branches?

These are not questions easily answered. There will be anecdotes aplenty regarding the lack of warning communicated to a wide range of these decision-makers. The first that comes to mind is the ill-fated CEO of Washington Mutual, who was allegedly incommunicado aboard a flight while the most significant transactions in the firm’s collapse were being finalized. This mirrors the earlier circumstances of the CEO of Bear Stearns. While a certain level of plausible deniability may be key to these positions, one wonders what kind of intelligence support these executives enjoyed.

Likewise, if warning was to be issued to an identified group of executives, who would have been responsible for giving such warning? Only a scant handful of the firms involved in the recent waves of disruption could be considered to have a dedicated intelligence function. Of these, few were likely oriented towards a warning posture, as opposed to the many other intelligence functions that constitute the duties of privatized shops within modern enterprises. Among the commercial consulting intelligence providers, the problem can easily have been defined by the lack of articulated customer requirements, and the lack of access and expertise that clearly prevented a more sophisticated appreciation of ongoing events. And one can question whether a warning account focused on what was largely a domestic financial market – despite the dramatic international implications – is at all a proper role for the intelligence community (at least in the United States). Certainly, as it is currently structured, it is nearly impossible to address – and no homeland security function has ever envisioned market shocks as a component of critical infrastructure protection. More damningly, the insights which would have unlocked these mysteries were not secrets to be stolen, but lay in perspectives which were never cultivated.

Again, there are likely case studies to be found in the after action reviews of the wreckage. Lehman Brothers, among the first to fall, most famously hired a former Deputy Director for Intelligence out of CIA to head its Sovereign Risk shop. But the structure and focus that geopolitically focused shop appears not to have been relevant to the manner in which the current crisis developed. Given that Bear Stearns itself allegedly was a leader in providing analytical research and other intelligence products to its investors and clients, the dissemination of these products to the executive level is worth exploring from more than an academic perspective. One can likewise point to other intelligence functions on the Street and elsewhere, stovepiped for threat analysis or market research or technology investment.

What few warning shops which may have existed to cover the sector likely followed the dominant paradigm of competitive early warning, focused on their competitors’ actions, positions, technologies and blind spots rather than the wider political and financial situation. The required optic was simply too large for most shops, whose production is typically serialized in daily or weekly form, no matter how strategic they might otherwise claim to be.

In sum, can one then consider this a failure of warning? There are no simple answers, and we certainly believe that this question will be revisited for years to come in future studies of intelligence surprise. The underlying causes are complex, but are clearly rooted – at least in part – in the lack of systematic warning intelligence coverage of the issues. Whether it was the role of warning intelligence shops to cover these issues is open to debate. However, this may be as much the result of the failure of a warning paradigm developed for a time and place now forever changed. One may liken this change to the decreasing relevance of the traditional state based indications and warning model, now replaced by the emerging strategic reconnaissance paradigm being explored at the cutting edge of the tradecraft.

There are also signs that this is far from over, as we move from the weekend into another turbulent week on the Street (and in financial centres around the globe.) While we may arguably have seen a strategic warning failure (or not), there is still ample need for operational and tactical level warning as the crisis continues. This need creates new opportunities for both the rare successes and failures that will make or break firms and fortunes. Unfortunately, it is exceedingly difficult to surge warning assets to these kind of non-traditional accounts in short order – particularly given the scope of the political, regulatory, and other uncertainties which plague the markets. This is a unique time – and a unique problem set – that will task the professionalism of involved intelligence practitioners beyond measure, given the excessively politicized atmosphere surrounding the issues. There are many intelligence professionals now treading virgin ground, far past the last signpost reading “HC SVNT DRACONES”.

We are reminded of Joseph Nye’s comments about that terrible day seven years ago: “September 11, 2001, was like a flash of lightning on a summer evening that displayed an altered landscape, leaving U.S. policymakers and analysts still groping in the dark, still wondering how to understand and respond.” Lightning has struck once again in New York, and again without effective warning. We expect the impact to the intelligence community – particularly the community beyond the traditional wheel of the major agencies – will be in its own way as profound.

Deliberately ignoring the human terrain

We have rarely seen such a work of profound analytic fallacy as the now much circulated study “Baghdad nights: evaluating the US military `surge' using nighttime light signatures”, which has been making the rounds throughout the blogsphere as of late. This paper purports to declare the Surge a failure based on the lack of increase of overnight artificial lighting, as measured by the Defense Meteorological Satellite Program (DMSP) sensors.

This sensor data has previously been used to illustrate the profound gaps between the quality of life in North Korea, when compared to the prosperity and wealth of the South. Electrical usage can generally be considered a proxy for economic activity, particularly in areas where public utilities must be augmented by private generation capacity.

The Environment and Planning paper provides night lights data only to December 2007. And while it briefly displays intensity mapping of sectarian deaths in Baghdad area neighborhoods, it largely ignores the decrease in such violence as the final outcome of “a vicious process of interethnic cleansing” rather than the result of the change in US counterinsurgency strategy and force commitments which was the surge. This is an assumption which cannot be supported merely through imagery analysis.

Needless to say, such an assumption ignores much of the literal reality on the ground – valuing remote sensing over the contemporaneous and local accounts of human sources, military commanders, and reconstruction agencies that have lived through the tumultuous progress of the latter stages of the Iraq intervention. It also conflates economic indicators with stability and security – a fatal assumption that invalidates any conclusions that might be drawn; an observation even an entry level intelligence professional would be expected to note. Further, one might very well question reliance on the relatively low resolution DMSP data for assessing complex urban terrain, particularly given that electrical availability has been a key topic of study for reconstruction planners. At the very least, a comparison of DMSP data against this ground truth baseline would seem to have been required.

Unfortunately, this is the very model of politicized intelligence; a study designed to create a single outcome through the selection of data it chose to present. It is also a profound argument against recent attempts to crowdsource analysis tasks to those who are not intelligence professionals by trade or training.

Against this approach we would cite a far more useful model for integrating the work of GEOINT professionals with that of ongoing counterinsurgency and civil reconstruction efforts, first offered by the National Defense Intelligence College. The paper, “Registering the Human Terrains: A Valuation of Cadastre” offers a far more productive means by which remote sensing data may be used to assess ongoing operational effects in a conflict theatre.

The night lights study is instructive, if only as a teaching case to illustrate the kind of error that self-reflective practitioners must identify and avoid in their own work.

h/t Creative Class

Forecasting through games

There is a long history of modeling, simulation, and gaming within the intelligence community, dating back to the Prussian General Staff’s Kriegsspiel, wherein the intelligence of the day, such as it was, would be used to determine the enemy strength and disposition to be set for the initial conditions of the map board. (An early American adaptation of the wargame – itself arising out of the intense interest in military professionalization in the latter half of the 19th century - can now be found in the digital stacks. It is worth a glance for those inclined toward matters historical but lacking either access to the original text or the German language skills with which to comprehend it.)

One can trace a direct lineage from such explicit gaming structures through the modern evolution of many forms of exercise and drill. Such efforts are increasingly reflected in new training and education efforts within the IC, such as the recently publicized virtual incarnations of several analytic exercises at DIA. The exercise materials themselves have a far longer history in more prosaic incarnations. The tanker war exercise that is the heart of Vital Passage, for example, has been used for teaching analysis of competing hypotheses for years using nothing more than paper and pen. The new immersive formats are clearly of value in capturing the attention of those students who have not yet been caught the more abstract means of envisioning crisis. It also serves as a good transition towards the application of the methodology in more complex, non-deterministic problems – particularly given the new emphasis on using assistive software to help track larger scale issues. (We unfortunately continue to encounter a number of younger analysts –products of the civilian university - that are unable to distinguish between ACH as an analytical methodology and the software used to automate that methodology. But that is another matter, and points to a failure of instruction at certain institutions rather than flaws in computer aided analysis or exercise).

For too long, though, gaming has stagnated essentially unchanged from its earlier incarnations. It has been left to the jesters and the speculators to push the boundaries of the tool, pointing the way to new directions and new uses. The most provocative of these suggestions – as is frequently the case – came from a jester at the futurists’ court, examined the potential utility of an alternative reality game structure as a recruiting and coordination mechanism for HUMINT operations involving unwitting participants.

A more immediate implementation has also now appeared, attempting to use massively multiplayer structures for long term analysis challenges. The Institute for the Future will launch its new project, Superstruct, on 22 September, which will attack what appears to be a catastrophic scenario using an alternative reality gaming architecture for distributed participation. It is a unique approach, described further through a FAQ here, and we can already see the benefits that the transparent and free form ludic design brings to the table. (We would note this to be a distinct difference from other crowdsourced analytic projects that we have recently seen attempted). We also have high confidence in the intellect and insights of the team that is executing the Superstruct project, having followed their work for some time, as well as having attended a fascinating discussion with other ARG designers from the original "I Love Bees" team at a Second Life salon hosted by The Electric Sheep Company a number of years ago.)

We have long been on record as highly skeptical of the efforts to use the intelligence community as the instrument by which to assess the uncertainties of future climate change, and have debated the issue with others of discernment who hold differing views. Yet the IC responded to the requirements levied upon it by Congress – as it always should. The resulting assessment, and public testimony, is a model of intelligence professionalism in the face of intense politicization. We find Dr. Fingar’s responses during questioning – clearly outlining the uncertainties of the scientific data, and the limitations of the IC’s resident expertise on the topic – a perfect teaching example of effective intelligence communication.

We think however that efforts such as Superstruct may be a better venue for exploring these questions, at least until the window of likely impact falls within the long range horizons of the intelligence community’s estimative views – be that fifteen, thirty or fifty years hence. It is also a fascinatingly cross-account and interdisciplinary issues – as well as a frankly lower priority intelligence problem – that is perfectly suited to experimentation with new analytic methodologies, novel analytic outreach, and new distributed production models.

We wish the project good fortune, and look forward to the after action assessment for any lessons learned that might be applied to future analytic tradecraft.

h/t Smart Mobs, and the Business & Games Blog

Wx-ing historical

We have addressed the recent fad towards addressing climate change as an intelligence issue several times over the course of this blog. We still remain convinced that while weather intelligence – Wx – will always remain a key factor in many accounts, climate change as a long term issue is simply beyond even the horizon that can be expected of the deepest of futures intelligence looks.

We are apparently in the minority in this view. We recently acquired a copy of an unclassified 1978 research paper from Central Intelligence Agency’s National Foreign Assessment Center (now republished by the University Press of the Pacific in 2005) which examined this very issue under the title “Relating Climate Change to its Effects”. For those younger analysts unfamiliar with the misty back history of old bureaucratic battles and therefore older acronyms, NFAC was the renamed Directorate of Intelligence (DI) under DCI Turner in 1977 – a designation which lasted only until reorganization under DCI Casey’s tenure in 1981.

We think perhaps this product might have best been buried with the old name. As a paper, it is almost entirely uninspiring – a mere 8 pages of substantive text, followed by hundreds of pages of tables and statistics that are the hard copy rendition of a contemporaneous data tape, mostly consisting of temperature and precipitation measurements assembled by a university contractor on behalf of USDA. These form the inputs to a simple climate model that was intended to provide for long term predictions of weather effects given specific outcomes, such as global cooling - a key concern of the day. (However, to their credit, the designers did examine the potential for global warming as well – which speaks well of the analytic rigour of the DI under any name, even if the paper is mute testament to just how badly scientific and technical experts can be at communicating with their readership through written intelligence products.) A speculative product such as this can be expected to offer no real conclusions – rather simply serving as a possible set of boundaries within the uncertainty space of future scenarios. However, it might well have made more explicit the effects it claimed to consider within the range of those scenarios.

But again, this serves to illustrate both the waste and the foolishness of attempting a futures intelligence estimate so far into the out years. We are not issued crystal balls when we are granted entry into the profession. It also serves to illustrate the perils of the arbitrary application of quantitative analysis as a fig leaf over unsustainable judgments. The model – no doubt painstakingly assembled and hard fought at the methodological level – is by its very nature the product of 1970’s era computer science. In the face of Moore’s Law, it is therefore over 20 generations obsolete.

Let us hope that this little musing upon the history of the account gives at least slight pause to modern practitioners seeking to enshrine climate change as a permanent account for long range intelligence analysis. At a point in time when supercomputational problems that take longer than a few months run time simply are not run at all until the next generation of architecture advances its inevitable order of magnitude or more, it is after all more than a bit presumptuous to assume that any community entity would be able to beat or even match the kind of big iron thrown at these problems in the civilian science world. It also very much begs the question of what better use such resources might be put to for other intelligence accounts – perhaps in the classic roles that the IC has always employed supercomputing resources: cryptanalysis, automated signal processing, or even exploring the new boundaries of potential offered by quantum intelligence.

If nothing else, this bit of history has also more firmly reinforced our opinion that climate change issues are a matter best left to the academics. Perhaps once the Long War has been won – and given the timescale we believe will be needed to accomplish this monumental, generational task – then the community’s attention can turn more to the matter once again. And if the current crop of speculative forecasts prove correct, at that point in time the issue may properly fall within the window of an actionable long rang estimate.

The perils of arbitrary and false precision

We find quite unhelpful the recent academic obsessions over estimative language – largely an exercise in the introduction of a numerical system which offers a degree of apparently comforting but entirely arbitrary, and therefore utterly false, precision. It seems however that we are in one of those cycles which seem to come along in the intelligence community every few decades or so, in which the numerologists and other soothsayers attempt to reshape the profession into their own desires for a more “scientific” practice.

Let us be clear. There are times when quantitative analytic methodology is vital – but there are far more situations in which it is misapplied, misunderstood, and entirely out of place. The latter comprise the vast majority of scenarios in which analytic tradecraft is called upon – not the least of which may be attributed to the highly unbounded and indeterminate nature of the problems with which we must grapple. And any time in which a quantitative basis has not been established, the insertion of numerical percentages for predictive purposes is little more than a farcical exercise in arbitrary selection. Over time, you may attune a group sufficiently in order to calibrate its judgment of these percentages in such a way as to create a consistency within that shared hallucination. However, this does not alter the underlying fallacy upon which such a house of cards is built. This is clearly shown in the number of cases in which the naive predictor is a better estimate of potential than the much vaunted group of experts’ judgment. Thus even in finance, the most precise of arenas, built upon the foundation of values, you will find predictions expressed equally alongside hedges – and the market littered with those who have failed to impose arbitrary figures on a highly indeterminate problem.

One of the greatest challenges in intelligence analysis is to understand the limits of prediction when going about the hard business of estimation. That understanding should shape the analyst’s focus on what ought to be examined for predictive possibility. These are, properly: the scope and nature of trends, drivers, and future scenario outcomes – and not the capricious shadings of difference between mathematical expressions of probability.

Estimative language has not been expressed through probability percentages for the sixty plus years of the intelligence community’s modern incarnation for good and well contemplated reasons. While the abstraction of the clean and sterile realm of mathematics is often a welcome change from the messy and hard realities of intelligence, that abstraction too frequently is used as a shield and an intellectual refuge for those unable or unwilling to embrace the challenge of actually doing intel.

Scientisim in intelligence analysis is a particularly seductive heresy. It offers the false promise of greater insight, should only additional efforts be applied more systematically, more rigorously, or with more and better data. But it has not been given unto us to see the future – no matter how carefully we might craft our equations. We may simply chart the boundaries of its outlines, and discuss the implications within the uncertainty space so described.

We have no doubt that we will revisit this discussion in short order. For now, however, we would close with an excellent reminder of the vast gulf of differences that may be concealed with that change of a single degree of significance in numerical expression. Originally produced for IBM, this admittedly dated video still serves to explain the staggering concepts of scale in a world of large numbers. (h/t to Thoughts Illustrated for pointing out its online incarnation.)

Questions of legality of intelligence in the commercial world

We frequently encounter those in the academic and business world which have little grasp of the applications of intelligence to the commercial world. These problems have not been helped by the distractions of discussion regarding economic espionage, directed by nation-states against particular industries. While the latter crimes do occur, and frankly have since the days of the first corporate entities of the East India Companies (Honorable or Dutch, take your pick), modern competitive intelligence is however a far different animal. And while any corporate entity may itself also commit a crime, there is quite a gulf between the examples that detractors might point to, and the standards which a profession establishes for itself in order to inculcate in its practitioners. Herein lies the heart of the frequent disconnect between intelligence professionals and their counterparts on the other side of the boardroom table.

This is not a new phenomenon, by any means. It has been our contention that the entire practice of commercial intelligence has a long and storied history that has been given short shrift, largely due to the influences of consultants and others seeking to brand their names onto this “new” thing that the re-discovered. While this may be a good way to sell books, and to organize conferences, we feel that it has greatly hurt the profession’s development and legitimization. Like national intelligence in the interwar years, it is too easy to wrap up a “new” experimental function like a business intelligence / competitive intelligence unit, particularly when budget cuts come rolling around. The taint of controversy has too often provided convenient ammunition as cover in such situations. Unfortunately, the very defense against these largely manufactured controversies is the same history that has been so deliberately discarded in favour of the branding of novelty.

Thus it seems we are doomed to watch that fight replayed over and again in the gulf of ignorance that has been the result. Yet the legitimacy of these units has long been settled law, among the oldest questions in the profession of intelligence to come before the bar. We cite a 1916 publication by the United States Bureau of Corporations, Trust Laws & Unfair Competition. (courtesy of the Government Printing Office), which references the matter (for American law) thusly:

Before entering the final decree in United States r. American Tobacco Co. et al the Circuit Court for the Southern District of New York considered a request to enjoin the defendants "from espionage on the business of any competitor, from bribery of employees of such competitor, and from obtaining information from any United States revenue official." Lacombe, J., denied the request, saying: “Why any one Individual or corporation engaged in this business may not acquire such information as he or it can legitimately obtain from private or public sources as to the business of a competitor we fall to see. When illegitimate methods are proved, they may be dealt with.”

We would wish to see that opinion more widely promulgated and discussed, in order to defeat the pseudo-legalistic arguments that have lately come into fashion to justify short sighted decisions that managers are unwilling to take responsibility for themselves. We think that such a changed calculus may actually alter those decisions in a marked way – or perhaps at least prove the unsuitability of certain ill informed individuals for the roles given to them in the complex decision-making environment of the modern corporation.

The loss of the history of the intelligence profession has had terrible consequences to its practice and evolution. It is long past time serious scholars begin to address these failings, in a manner that advances the literature of the field in areas of direct relevance to the practitioner.

Rent seeking and digital media exploitation

We hear news of one of the more disturbing aspects of legal intrusion into the intelligence field – the tendency of lawmakers and regulators to craft protectionist legislation – enabling and encouraging rent seeking behaviors that would never survive in a more open marketplace.

This time around it is an area which at first blush might seem tangential to the profession of intelligence - but it is more closely associated with other issues in the field than we would like. South Carolina may force those private parties involved in digital forensics to hold a private investigation license – that hoary old document so beloved of fiction authors and scandal page writers.

The PI business has clearly been losing a lot of ground over the past few decades as business has become ever more global in nature – and not that it ever had much ground to begin with, in the more rarified professional atmospheres of major multinationals. The few top tier exceptions in the field have undergone some dramatic shifts – so much so that reporting on the industry’s turmoil became a staple of the Indigo Publication’s Intelligence Online newsletter. Major players from consulting firms and more specialized shops have emerged to carve away what might have been business opportunities, especially since most executives (and more critically, the lawyers who advise them) disdain the “hard-boiled” image that the term PI evokes. Even if in the modern business of investigations such an image is only a stereotype, it has a strong impact on business realities.

Digital forensics is a pretty wide field, even despite its relative youth. It encompasses activities as disparate as internal systems monitoring, to compliance auditing, to the full range of criminal search and seizure. What it means in a networked world is still very much being explored – particularly given the challenges posed by rapidly evolving hostile technology innovations. In its more advanced networked forms, it often begins to resemble more closely activities in the signals intelligence domain. While this is understandable from the perspective of intelligence studies theory, given that the native competencies of intelligence in the cyber domain have yet to be recognized, it is far to early to allow regulatory frameworks to disrupt the development of the discipline – especially a framework which imposes mechanisms better suited to the Industrial age of rail than the modern age of networks.

We oppose this line of thinking not only for the immediate harm it causes, but for the slippery slope it creates. With each new regulatory overreach, additional activities within the intelligence field become new marginal examples that could be brought under one’s favoured framework. From digital media exploitation it is not far to also seize upon the regulation of document exploitation, and from DOCEX it is easy to bring other aspects of OSINT under the sway of those seeking to profit from entrenched positions. How long then before simple searches within a database, or simple overt elicitation contacts, will require a licensing regime – and one ruled by a clique of established players with only the "right" kind of political connections?

This is among the reasons why we have also opposed the creation of arbitrary standards bodies seeking to define professionalism within the intelligence field – especially those composed of academics who cannot even properly conceptualize many of the key aspects of the profession as it is actually practiced, let alone standardize external measures by which practitioners might be judged. We hold no objection to bodies seeking to advance professionalization through a community of interest, a strong literature, and leadership by example. We also see no issue in individual standards being put into place in specific circumstances – such as the conditions under which a court will recognize sufficient expertise for testimony, or how a government agency will certify knowledge and experience ample for the performance of a contract it is to award, or how employees will be evaluated for promotion within a given shop. But these are far different things than the regulatory creature now rearing its ugly head.

We hope that some degree of sanity will return to these discussions, particularly should the lobbyist guns of the major consulting firm players be warmed up to play against the bush league minors that appear to be driving this process. While the South Carolina case is but one state, the precedent could create more widespread attempts at similar rent-seeking behaviors, both geographically and down that slippery slope of further overreach.

h/t Slashdot

Arguments in intelligence history

We occasionally note the scrum which emerges as the topic of intelligence history is approached within the context of larger political science debates. In this case, Sic Semper Tyrannis has resurfaced the debate over the utility of social science derived approaches to intelligence analysis practice; centered around Sherman Kent himself and the original Ivy League influenced R&A, first raised back in 2005. The current piece expands the discussion into a wider examination of the use of intelligence within a supposed political school of thought allegedly shared by several recently prominent decision-makers. Though we find the formulation “neo-con” to be a particularly unhelpful analytical framework, both too heavily influenced by recent political rhetoric and prone to use as a shorthand masking too many anchored and unexamined assumptions; the piece does offer some value in summarizing a set of arguments of the contemporaneous back and forth of Soviet analysis that is rarely well captured in modern study. Also quite interestingly, whatever one’s take on the issues themselves, the discussion does illustrate the impact of the academic and privatized intelligence tradition on key community history, influenced which are too often overlooked and thus all too rarely taught.

We also note the very differing perspectives on the first exercise of alternative analysis, the famous Team B studies, to other views about the experiment referenced by Former Spook over at In From the Cold. It’s an old stalking horse of the intelligence academia, and in our opinion, too much of the subsequent debate over these cases has been influenced by contemporaneous political maneuvering intended to protect a government monopoly on intelligence in the face of a unique privatization challenge. We note that alternative analysis has now been enshrined into community practice – thus the lessons of the original competitive analysis exercise could not have been nearly so negative as many subsequent commentators would insist, whatever one might believe about the relative estimative accuracy of each team’s conclusions. Tradecraft may well be advanced by even imperfect experimentation, so long as methodologies are refined in subsequent iteration.

We also wish to thank Non Partisan Pundit for the kind words in the comment thread, and hope we may continue to strive to earn the appellation “not only well written and informative, but it's also refreshingly free of politics.” An intelligence practitioner is always intended to be professionally apolitical – a fact that we believe is not stressed nearly enough to the younger generation in an environment of highly politicized media, and the ongoing football between the Hill and the Administration (and back again). We see not only the ethical obligations inherent in this requirement, but also the practical benefits to the apolitical approach, as exemplified in Tetlock’s research into expert judgment, best approached through his recent work cast in Berlin’s metaphor of the fox and the hedgehog. We also look forward to the Pundit’s further examination of intelligence studies issues, such as the discussion of the effect of “yes men” within closed regimes.

Understanding intelligence consumers

One of the persistent myths that we find too often in intelligence shops – particularly among editorial and production management staff – is the idea that an intelligence consumer cannot internalize complex concepts and ideas, and thus requires extreme levels of simplification. Where exactly this myth first originated is lost to time, but that it continues to dominate the thinking of those who ought to know better is perplexing to say the least.

There is a vast difference between simplicity of expression and complexity of thought, and all too often the search for the former renders incomprehensible the latter. One sees this especially in products which emphasize the removal of common terms of art – despite the utility such short phrases bring to both consumers and producers when referencing a body of other works. The use of such terminology does not excuse the author from placing them into a relevant context, and using such context carefully within the structure of the products argument. However, all too often editors and others involved in coordination reject such expressions of brevity with blanket dismissal, citing perceived military origins of a phrase in many cases (as much of intelligence’s activities derive from war fighting origins, no matter what the current application), or as “excessively” technical. This is a grave disserve to the reader, who if they are not acquainted with a key concept associated with an intelligence issue, should be introduced to that background rather than left to assume understanding of a distorted turn of the phrase.

More and more, the results of this tired old trope become a primary reason why many consumers cite finished intelligence as irrelevant to their actual responsibilities – all the moreso when those consumers hail from segments of the community’s readership that may typically not have previously relied upon finished intelligence as part of their decision-making processes. Given the increasing levels of education demanded of even middle management, let alone the kind of experiences a typical senior might lay claim to, it is understandable that they would reject any attempt at intellectually coddling at the same time they are expected to be informed by a product. After all, it is not uncommon to be briefing or writing for a consumer with post-graduate education and longstanding experience grappling with difficult legal, business, and public policy issues. And given the number of lawyers that now make up the ranks of senior government and industry leaders – it is difficult to insist that an individual who might in another context be expected to rapidly skim and comprehend the most challenging of legal briefs must somehow never be troubled with a product written at a level higher than the typical newspaper’s 8th grade education standard.

This also contributes, in our view, to the reasons why consumers typically emphasize the value of many forms of “raw” and other internal intelligence reporting. These latter products tend to be authored for the primary readership of other intelligence professionals, and is therefore not “simplified” in the fashion one comes to expect from the editing and production process. It is no wonder then that many consumers – in some cases now more experienced than the analysts and other staff which serves them – prefer the “roughs”.

In the end, the manner in which a finished product is best crafted demands that the author understand the reader – and not simply rely on old maxims of some journalism editor’s style guide, applied without real consideration.

Virtual talking heads

The circulation of the survey instrument for Mercyhurst’s new research study on video format delivery of finished intelligence has occasioned no small measure of debate among participating professionals within the community. While we are glad to see the research being conducted to advance the understanding of the field, we have some serious questions about the wider implications of the use of video for product delivery. Its historical applications have occasionally resulted in unique value – particularly for certain high level consumers, such as its famous use as part of President Reagan’s daily briefing. However, for routine production it imposes burdens on development, editing and coordination, archival processes, and technology / connectivity requirements that create issues far beyond the initial considerations of communications efficacy.

The entire discussion also reminds us of the earlier attempts to re-create intelligence briefings into virtual formats through automated processes. First popularized in response to Web 1.0 start-ups such as the virtual news readers such as Anna Nova, the concept never moved beyond research and development phases precisely for some of the same reasons. The removal of the human briefing element also created a disconnect between attention and pacing, which alienated many consumers whose areas of interest diverged from the set piece delivery – but who may have been more productively engaged by a dynamic briefer. There is also the subtle but very real impact of the uncanny valley – the noticeable aspects of the artificial in “too human” like virtual entities. (We wish to note one of our close colleagues’ wise but whimsical suggestion to replace a human virtual briefer with an object of classic children’s theatre. If nothing else, we would love to have such a product on hand to inject a bit of levity into meetings in which the participants take themselves too seriously.)

Set piece delivery tends to work best in environments where the consumer can devote only partial attention to the product – such as audio delivery of briefing or lecture materials, designed for those who are looking for cognitive engagement during other routine tasks such as driving, physical exercise, or the like. We have not yet seen an environment in which video delivery could meet this same niche – particularly for products which demand a higher degree of attention than watching CNN with the sound turned down in the typical watch center fashion. Virtual talking heads suffered the same problem.

There also remains the serious question of the longer term impacts of removing one of the few opportunities for direct interaction between consumers and intelligence professionals, where the intangibles of human relationships are nowhere as easily quantified. Perhaps there are some aspects of intelligence that simply ought not be automated, to the ultimate benefit of consumer outcomes.

Nonetheless, we will be interested to see the results of the study, and happy to see the literature of intelligence advanced in this area. We can only hope other academic studies will continue to build understanding of the nature of intelligence communication.

Revisiting the case method

We have long been proponents of the Intelligence Community case method for teaching the art and science of intelligence. This is hardly a controversial position – though it seems to be one little implemented in many academic institutions, where it could do a great deal of good. Even those schools engaged in writing cases for the community do so rarely practice what they preach (or at least contract to promulgate).

It is thus with interest we note a new paper examining the use of various case study approaches within the legal profession. While the various scholars in the intelligence studies field would have the teaching of intelligence emulate that of other professions, such as law and medicine, these models might have a profound impact on what are currently successful methods. We have been on record as objecting to the too close commingling of the behaviors and thinking used in the practice of law with the tradecraft of the community. In this study, we might also have found reasons to sustain an objection to the adoption of legal teaching methods in the form that currently dominates the institutions before the bar.

It should be noted that the original IC case method program as developed within OTE began out of the model of the Harvard Business School, rather than the law school side. Other institutions seeking to emulate their success would be well served to consider this.

We also agree with the Barton paper’s focus on the importance of exercise and simulation in going beyond the theoretical discussion of cases and into the practical applications of that learning. There is no substitute for experience, but exercise environments can provide the first shadow of such experience for the green intelligence professional.

Understanding the evidence of events

A few items have continued to trickle across the nets which bring the all too often forgotten works of AFRICOM to our attention.

The first of these comes from the ever perceptive folks at Coming Anarchy, who note the continuing engagements involving host nation and US forces tracking irregular forces in the wilds of the Sahel. The Taureg rebellions in Mali and Niger are in one sense the very real stuff of which 19th century adventure novels were written, yet at the same time they are the concrete manifestation of the Global Guerrilla / 4GW+ problem set in a small, poor, and out of the way place in the world. A place that just happens to border key areas with other known problems also involving restive populations heavily influenced by militant Islamist ideology….

More troublingly, though, we also note the Economist’s puzzling reaction to the body of public press reporting regarding Islamist – and in particular, Al Qaeda, activities in the region. They editorialize in an article on the dynamics of the rebellions: “But apart from the odd smuggling deal over guns, drugs or cigarettes, no solid links… have been established”. This despite continued open source information regarding the continued expansion (or retreat, depending on one’s perspective) southward of the Salafist Group for Preaching and Combat, who have re-branded themselves as the Al-Qaeda Organization in the Islamic Maghreb. (And while we may not always agree with their analysis or policy recommendations, International Crisis Group has published to its usual standard of excellent research regarding these issues, as has the inimitable Douglas Farah – perhaps the first journalist to seriously cover the lesser known transnational aspects of global terrorism networks operating in Africa.)

We find a persistent form of cognitive bias in this sort of thinking as demonstrated by the Economist, which would deny the evidence of events. It is one that has increasingly come to dominate policy and intelligence related discussions in many institutions. Let us call it the dissociative fallacy.

The dissociative fallacy is the tendency to ignore linkages between issues based on pre-concieved ideas regarding how such linkages manifest - ideas typically based on unarticulated assumptions, and frequently informed by popular stereotypes and other fictional information. It rejects the actual form of associations when they do not meet an idealized a priori postulate.

The dissociative fallacy is a natural development for those that perceive their role to be primarily centered around minimizing disruption and managing crisis, particularly as those acting in such roles often encounter deeply entangled issues which are not amenable to solutions in part. It is understandable, if not acceptable, that they should then seek – consciously or unconsciously – to reduce the number of factors at play in discussions regarding the issue. But it is a dangerous way of thinking that just as surely leads to intelligence failure and intelligence surprise as any other form of cognitive bias.

We see this fallacy at play in counterproliferation discussions, used to explain away any given set of dual use or even prohibited materials as unrelated to a weapons program end use. The thinking goes that unless it the material under examination is part of a “stockpile” – with attending implications of bubbling vats and ranks of missiles arrayed in Soviet era precision – that it shouldn’t count. (Something all too common in the post OIF era, as the “lessons learned” of the pre-war estimates continue to be applied before the final judgment of history – and the full accounting of the regime’s activities during the 2002-2003 time period – have been reckoned.)

We see it frequently cited in counterterrorism cases, in which travel linkages, communications exchanges, ideological references, and self-identification are discounted in favour of theories of self-radicalization and explainable mental disorders. And for as much of a threat self-radicalization does actually pose to permissive democratic civil societies, all too frequently in recent experience substantive associations to more organized terrorist entities have emerged during post-incident investigations of supposed “lone wolf” actors. But since most of these relationships do not fit the Hollywood-driven assumptions of how terrorist associations are formed, they are unremembered until after the next attacks.

The fallacy itself is perhaps most prevalent in other transnational issues area, where single case study examples of an incident type are merged with implicit assumptions regarding organized crime or other threat activities, creating a “standard” model which does not begin to encompass the range – and frequently, the strangeness – of real world cases.

For as commonly as it appears, there is very little in the literature on this point. There is a clear need for academic research in this area. Perhaps some bright young thing from one of the better institutions might taken an interest, and grace us with unique study and innovative insight into the nature of the problem.

Markets for prediction versus prediction from market data

Our readers will recall that we have long been fascinated by the potential for new concepts in analytic methodology which might allow the imperfect instrument of predictive intelligence to be refined and applied under new conditions. Among the vehicles which have emerged over the past few years that might seem to offer promise is the idea of the prediction market – instantiated in various forms under DARPA, Long Bets, and other iterations.

We have yet to see any truly definitive examples of such a market’s utility to the practical and very real problems of forecasting for intelligence issues. We are also increasingly mindful of the new issues which emerge during the implementation of such markets – especially those related to strategies which successfully game the market from the perspective of an individual “trader” (responding correctly to market conditions to create “wins” for his own portfolio), but which degrade the pure anticipatory value of market based information due to meta-strategies focused one the market itself rather than the issue to be predicted. In this, we recall Nassim Nicholas Taleb’s admonition regarding successful trading strategies that only require one good year in a century to produce a profit, and who insulate against major losses through a strategy of slow cuts.

We thus were initially quite interested in a new paper which attempts to evaluate the Surge through the lens of a prediction market approach. However, the key difference we note is that the paper is an attempt to measure not a market consisting of predictive judgments, but rather to derive predictive value from a market of financial judgments. In our view, the attributing causalities to the decisions of finance is a far more difficult business, and so prone to errors of bias that we would question its practical utility.

However, it is an interesting example of the difficulties of cross disciplinary applications in the intelligence space. (And a clear definition of why intelligence is an art and science worthy of study of its own, as opposed to merely a bastardized aggregation of various social sciences – as some critics have in the past charged.) We encourage such efforts, not because we hope to gain transcendent insight on specific issues through any particular experiment, but so that we can continue to incorporate new approaches into the business of intelligence through trial and error. Identifying a dead end pathway is as valuable as finding the next new thing.

To this end, we would also recommend our readers to the excellent item at Mapping Strategy regarding the perils of actuarial approaches to prediction, in which we would heartily second the call for caution in making assumptions regarding the predictability of events based on their historical occurrence. (This is one of the reasons we have been so critical not only of Schneier et al’s comments on counterterrorism issues, but also of Pape’s work regarding suicide attacks.)


h/t Marginal Revolution and Economist’s View

The role of intelligence producers in national policy debates

A new chapter of intelligence – policy relations has been firmly cemented with the release of the latest UBL message (no matter to what degree it was staged and altered in AQ’s propaganda shops). The fact our of most hated enemy has taken to citing US authors and domestic political debate talking points in support of his (and his organization’s) cause has been a disturbing development dating back to at least 2004. However, when the enemy cites the work of a former intelligence analyst due to the level of unseemly publicity that individual has attracted, it is time to question the role of the “retired” community members that now speak so frequently in national media. (This is to say nothing of the entirely contemptible leaks by intelligence insiders that have so damaged the country’s capabilities across its most sensitive programs in favour of transient political advantage.)

Some speak from positions of substantive experience at pivotal moments in history, some attempt to leverage marginal roles from decades ago into a platform supported by their association with the entity rather than the value of their ideas. The number of these individuals, and the impact on public debate, is an unprecedented thing in the history of the US intelligence community, even given the typically difficult tensions between intelligence producers and their policy level consumers. The business of intelligence deals often with inherently political subjects and judgments, and it is a constant struggle for those serving on the line to defend against politicization (of their own bias, or of the temptations of external pressure). Yet those who no longer face the constraints of the line seem often to become very political creatures themselves.

In one sense, this may be legitimate, were they seeking policy positions as office holders or even appointees. However, the endless circuit of talking heads and pundits that have emerged, feeding from the trough of ratings created by scandal, is another matter entirely. There may even be legitimate roles for a former intelligence professional in the media – the help contextualize and inform regarding complex events and situations, for instance – the lack of which has been a defining feature of the failures of the media’s coverage of many recent cases. But there is a distinct difference between discussion and advocacy, which is as clear of a bright line as any found in between analytical judgments and policy proponency.

There is in journalism the old adage that a reporter of the news should not himself become the story. Perhaps there is also a corollary that should be penned as similar admonition to those that serve (or have served) in the intelligence field. Our honours come not from praise in the public forum, but from the quiet acknowledgment of our peers over a lifetime of silent service that yields good outcomes which only history shall chart - long after we have gone to our graves.

The sins of analytic methodologists

We are strong proponents of improving the rigour in intelligence analysis, and have frequently advocated for the better grounding of new analysts (and in-service training) in formal logic, epistemological theory, and structured technique. At the same time, however, we are deeply mindful of the continuing unmet needs for more creative and imaginative analysis, which moves further afield from the narrowly constructed questions of current uncertainties and into the realm of the unknown unknowns.

We are thus deeply troubled to observe a frightening trend in the work of some methodologists attempting to “solve” the problems of analytic rigour through new tradecraft practices. There is an increasingly common conceit that reliance on the analyst - subject to, cognitive bias, information overload, and human fallibility – can be engineered out of the process of doing intelligence. Instead, certain methodologists would substitute organizational structures, workflow re-organization, and the introduction of supposedly superior quantitative metrics in order to create a new standard for “answers”. The underlying thrust of these efforts is to reform intelligence activities towards a more “repeatable” process, often described by industrial or scientific metaphors such “foundry” or “lab”. These typically originate from the engineering and technical intelligence disciplines, and are usually directed as criticism of typical all source efforts – particularly those grounded in social science fields or qualitative methodology. Among the recent examples of these attempts at “standardizing” the process of analysis – with new emphasis on technical and quantitative approaches – is the 2006 DNI Galileo program paper “Predictive Network Centric Intelligence.”

The fundamental flaw in many of these methodologists' efforts is that they are essentially reductionist attempts to force the difficult and oft-times messy art of intelligence entirely into the narrow box of its scientific side. While there is a place for scientific approaches, particularly in the grounding and validation of assessment, the inherently creative, non-linear, and even non-rational elements of the profession can never be completely discarded. Most recent intelligence failures have occurred, not due to a lack of precision in judgment, but from a lack of imagination in identifying, describing, and forecasting the uncertain dynamics and emerging complexities of fast-changing accounts.

Most methodologists are very good at creating shiny and intricate machines for solving puzzles to produce answers of intelligence “fact” – often with great and noteworthy expenditure of time, effort, and resources. However, the community is challenged most today by mysteries, to which there may never be a single set of answers but rather only approximations and options, set against the backdrop of an ever evolving understanding of situations obscured by the fog of war, time/distance, culture, and levels of commitment.

It is also fascinating to note that many of the methodologists reject individual level approaches to solving difficult intelligence problems, preferring instead to focus on the creation of new systems and processes. It is our contention that the analyst will always remain the heart of intelligence insight, and the things that happen inside the human mind are non-replaceable - and frequently non-reproducible – elements that cannot be dissected and pinned neatly for methodological examination. The analysis “bench” will never be deep enough to support the creation of large structures of process in which analysts become interchangeable parts. Analytic tradecraft can provide best practices to help avoid certain fallacies and other mistakes, and can demonstrate cases in which valuable lessons can be learned, but it cannot by itself produce the innovative and imaginative prediction that will help decision-makers address the kinds of challenges they increasingly face.

It is for this reason that we continue to emphasize the importance of the cultivation of the analyst above all else. We would hope that the methodologists would at least consider pursuing research, which rather than seeking to radically restructure the forms of complex intelligence activity that have evolved under brutally Darwinian pressures, will instead support the pursuit of excellence by individuals facing hard challenges in the face of serious consequences. The analytic house neither needs nor particularly wants shiny new puzzle machines – but is desperate for additional tools and techniques which can be called upon to address mysteries in times of crisis, and when other sources of inspiration have run dry.

The difference between intelligence, covert action, and policy

If true, this story illustrates why the machinery of intelligence and covert action should always be kept separate from the other instruments of policy and national power.

It is also one of those times when one looks back at the decision to involve DCI George Tenet directly in negotiations in the Middle East (in his titular capacity, no less) as the height of folly. After all, there are plenty of Arab conspiracies which attribute everything wrong in the world to that particular organization… and bringing them to the table by name did nothing but add fuel to that fire for generations to come.

In the same light, we look at the old Department of Justice efforts to train and equip PA police… we wonder how many of those individuals may have been executed in the streets in the past few days – or who are doing the killing themselves?

Nothing so blunts a nation’s capabilities than the use of those scarce resources for missions that other agencies and organizations should have done, but failed at. One can say the same risks lie in store for the “Department of Everything Else” long advocated by Tom Barnett.