Glimpses into agent psychology

We have long held the opinion that the British writers of the intelligence novel have offered the best examplars to be found of the psychological experiences of agents recruited for espionage. (That is, the classic intelligence definition of the term, and not the disastrous usurpation of the designation for the law enforcement community). Among the unique insights have been the often striking similarities between the subjects and those officers responsible for handling them, something particularly more visible in certain accounts and historical case studies than others.

We speak of course of Len Deighton, whose works remain classics, and of John le Carre, whose early pieces well captured the tenor of his time (despite the increasing gap between his later fiction and the realities they allegedly represent). A more recent entrant is Charles Cumming, whose depiction of the post-Cold War British intelligence establishment from the perspective of a partially witting asset is striking in its tone, not least of which is the result of the author’s semi-biographical approach. (This has been true of most of the better intelligence fiction writers).

It takes a certain personality to write fiction of this sort. Some insights are only the result of lived experiences. We are unsurprised, then, to hear recently of Le Carre’s early intent to defect to the Soviet Union. One can easily see the life-long hints of such intentions throughout much of his work, and most notably in the characters which in fiction carried out that which the man himself never did. It also provides an underlying cause to explain the often excessive intricacies found in the novels’ plot lines, which a number of years ago caused a former Russian intelligence officer to remark in despair regarding the negative impact that such fictionalization had upon younger professionals in his service attempting their own approaches along such models, without regard for the inevitable imposition of Mr. Murphy’s Russian equivalent in the real world. After all, after spending so long considering what one does not act upon, it is only expected that the resulting planning takes a tangled and impractical shape.

For students of intelligence as an activity and a profession, it is a subject perhaps best handled through fictional characterization. In this form, certain features may be exaggerated for the purposes of the narrative – and over time, a composite constructed to reveal the whole. Consider this sort of fiction perhaps the intelligence community’s version of the morality play.

We do wonder where the next century shall find the literature to play the same role. In this, we are reminded of the most insightful essay by Charles Stross, The Golden Age of Spying, in which he quite neatly characterized the unique cultural pressures which brought such works to the published market. We do not see a modernized parallel anywhere on the horizon, and we think the profession poorer for it.

Revisiting analytic rigour

The research currently being done at Ohio State University into the problems of intelligence analysis – including information overload, cognitive processes, and other aspects of the methodology – has from time to time caught our interest. Among the more interesting of these items now in circulation is an excellent lecture that we most highly recommend to our readers, recorded last year during the too often overlooked Google Talks series. The discussion focuses on the evaluation of analytic rigor, and means by which analysis may be strengthened.

We particularly favour the philosophy that Dr. Woods presents, which seeks to avoid dictating a single best methodology or process. We are more than willing to listen to the methodologists, but too often we find a dictatorial approach significantly at odds with the realities of line analysis. We think that the observed case study technique used in the Ohio State team’s research – something too infrequently done by many academics – is key to the validity of their findings. One cannot discuss analytic ideals without involving those who are actually involved in applying tradecraft to real problems. It is also not enough to conduct such research in artificial environments within student populations – real line analysis is too different.

We certainly cannot agree with the apparent off-hand condemnation of “folk” psychology of intelligence analysis – clearly aimed at taking on Heuer’s “bible”. While we think that there is a clear role for the methodologists and their research into strengthen analytic tradecraft, there is also a very real need for interdisciplinary adaptation from other areas of social science, as well as the kind of internal discussions that make up a key part of the maintenance of those oft-criticized, but entirely vital, guilds that are the backbone of the community.

We do however find several key concepts of great interest that deserve wider attention, including the concept of the Supervisor’s Dilemma – the balance of customer outcomes requirements and analytic resource opportunity costs against the relative depth of analytic rigour. We also find the study techniques themselves of interest, especially the concept of elicitation through critique – something we feel will likely have a far greater applicability in capturing the kind of intergenerational knowledge that the community is in danger of losing. We see the technique as one means of making more formal – and scalable - some of the kinds of subtle interactions that characterized the experiences of apprentice and journeyman analysts under the mentorship of a master.

We also find great merit in the good professor’s comments regarding the overconfidence of new analysts, and the satisificing biases that result. We definitely have observed a level of arrogance in too many new hires – and especially those coming out of the intelligence studies programs. The first lesson that an analyst student should learn is the fear of God – and of their own error. Too many programs of instruction are not affording the student the chance to learn that fear from the visceral experiences of their own mistakes, and to take away from the experience a humility that will cause them to productively question their future work toward its improvement. Such experiences are far better gained when the consequences are not fatal, in line with the lessons taught by Red Flag.

There is much food for thought in this lecture, as well as the contributions to the literature that the Ohio State program has generated. We will no doubt have further commentary on the subject in the near future.

Revisiting Twelve Angry Men and legalism in intelligence analysis

For a number of years, the classic black and white film Twelve Angry Men has been a frequent teaching aid in introductory analysis courses dealing with the basics of evidence and argumentation. The conventional use of the film is to provide an accessible means of deconstructing a fictionalized scenario for students with little prior experience with formal debate. Given the current decline in public education, this helps remedy a basic skills deficit that is unfortunately and increasingly all too common.

However, it is with interest that we observe the controversy that has erupted once again over this fifty-one year old movie. The criticisms that have been leveled against the underlying premise of the film deserve some additional consideration – not the least of which because they point out the serious problems in applying much of what is taught as legal logic to the unique problems of the intelligence domain.

The Spectator’s argument surfaces one of the reactions common among many students, but in a far more articulate fashion than any entry level professional might be expected to voice. In essence, this criticism is based on the need to focus on the external worldview, rather than the tactical maneuvering in the courtroom that such kinds of arguments inevitably devolve towards. It is a quite valid point, and among the reasons that we have long decried the trends towards creeping legalism that have lately come to dominate intelligence work. The bulk of a lawyer’s litigative activities – and therefore a disproportionate degree of their education and professional experiences – are dictated by entirely tactical considerations that apply nowhere else but within the limited framework of the legal system. Too often this is easily forgotten, to the detriment of the strategic picture – and the accuracy and veracity of analysis. We have written on these problems before, but to be frank we had rarely considered the myriad of ways in which – by borrowing from the older legal profession’s traditions in teaching basic logic and rhetoric – the intelligence academia may continue to contribute to these unhelpful trends of cognitive bias. Among these, of course, are the kinds of ludic fallacy identified by Nassim Nicholas Taleb.

The folks at Volokh Conspiracy take a different tack, arguing for greater consideration of interdependencies within factors under examination in the film’s fictional trial. This is also a very useful approach for discussions with students, many who likewise tend to view evidence in isolation. It is particularly appropriate when covering structured analysis techniques such as ACH – and one that rarely touched upon, if only due to the frequently too shallow examples offered to illustrate the methodology, which are unable to support a more robust discussion.

Westminster Wisdom rises in defense of the film, and illustrates the more important but also often overlooked value to the piece in the intelligence studies classroom – the discussion of uncertainty. The intelligence professional will always work within a framework of ambiguity, doubt, and frequently, deception. However, the role of intelligence is not merely to reach a lower standard of proof than that used in a criminal trial (or even the lesser civil threshold), as is commonly taught (and in particular, a tenant of faith within law enforcement intelligence). Rather, intelligence’s purpose is to provide accurate insights despite such uncertainty; and where absolute accuracy is not possible, to bound the space of uncertainties for the policymaker in a manner that supports informed decision-making.

In light of all of the foregoing, we continue to search for good alternatives to the film for use in the classroom. Our search is also driven by the simple fact that to the Millennial generation, the black and white format is very nearly entirely alien. It creates such a visceral negative reaction that the first ten to twenty minutes of the film are simply an orientation to the unfamiliar environment. The pace of the thing is also glacial by modern standards, and particularly so to minds attuned to rapid multi-tasking and immersive information environments. While one can make all the arguments one likes about the need for sustained single focus attention, their native preferences are indeed more suited for the kind of world in which they operate as intelligence professionals than the Industrial era conventions that black and white film represents.

The single set format, and the emphasis on argumentation, has however made it exceptionally difficult to find a substitute. Our best – but admittedly still imperfect – alternative has lately been the 1999 film Deterrence, which offered a President’s decision-making process in a nuclear crisis while snowed in at a small diner. Unfortunately, the film’s scenario is constructed around a fictionalized Iraqi dictatorship – which in the modern politicized climate often steers debate too far astray of the real purpose and into the debate over Operation Iraqi Freedom. It also pre-supposes a certain level of student knowledge regarding nuclear warfighting and mutually assured destruction strategy: something not always guaranteed in the post-Cold War cohort. This sometimes makes for quite interesting discussions, to say the least. These are the same problems that incidentally also led us to abandon using the old JMITC exercise that relied on the film version of Hunt for Red October as a notional scenario from which the students would develop practice analytical pieces. While, as a friend recently reminded us, that particular film ages very well as such things go, we recognize that writing intelligence on Soviet era ballistic missile submarines is an anachronism to which few students will respond well –and one that does not serve their real and current professional needs.

All in all, teaching intelligence with films – for as engaging as the technique might be for students bored with lecture – remains a difficult proposition. We hope that in time the development of newer tools for digital animation – and the kinds of interactive scenarios that new gaming engines permit – will render the question entirely moot. But the cultivation of young professional minds remains a terribly stubborn business, and one that is not frequently improved by new technologies. We do wish to see such improvements become effective, and preferably in the near future. But given that the same promises have been made since around the time black and white films first graced the big screen, we remain skeptical.


h/t Overlawyered

Critiquing Myers Briggs for the (new) intelligence workforce

Not so very long ago, it seemed that the Myers Briggs Type Indicator was all the rage within a certain segment of the intelligence community. It was imagined that one’s personality could be pinned neatly to cardboard, and used as a proxy for judging individual reactions by managers essentially uncomfortable with the messy realities of actual humans. What started as an interesting thought exercise, intended for new analysts to learn about themselves in the spirit of reflective practice (much like Johari’s Window and similar concepts) soon was extended far beyond its initial use into a whole range of areas that it should never have been attempted in.

Sometime over the course of the past few years we saw a brief respite in the recycling of the MBTI “discovery”, and thought the fad over. After all, there is a war on, and there were quite a few other new attempts at understanding the intelligence community’s workforce in a more sophisticated manner that had been discussed since. We had hoped never again to see other individuals wearing their type code as badge around the vault…

Clearly, we relaxed too soon. There is rumour of an as yet unpublished work now circulating within academic circles, which examined the MBTI of a cohort of intelligence studies students. It claims to draw some sort of generational lessons in how the Millennial will differ from their older counterparts in the community.

Now, we are among the first to point to the radical impact of the oil and water mixture of the newest members of the community with practitioners of a certain age and kind of experience. This is not in our opinion a bad thing by any means, and in fact we have seen it result in some fascinating and innovative outcomes that no one could have predicted in advance, least of all the participants involved. We also hold the greatest hope for the students yet to enter the profession, for they will be the true drivers of a kind of transformation never before seen in the community as the Long War comes to full term.

But in addition to our critique of the uses to which the MBTI has been bent, we would strongly question the utility of using a student population at a civilian institution as the benchmark for the cohort. After all, most of those students will never have spent a day within the IC (or even in a full time private contracting shop.) The behaviors and activities of a millennial college student are a far cry from their employed professional counterpart of the same age – particularly given variables such as military service. Having shepherded more than our fair share of new cohorts into the community, we are acutely aware of the changes that are quickly wrought in individuals as the weight of their new responsibilities settles full upon their shoulders.

We are also greatly aware of the changes in behavior and expressed personality characteristics created by the unique pressures of the Long War, which can vary greatly depending on the roles these individuals are assigned. Such changes demonstrate the highly situational nature of the indicator, as an individual assigned to highly collaborative collection tasks will express as far more outgoing and concrete on a particular day than he or she might when tasked to intensely cerebral, isolated research on another.

The student population also likely contains a high percentage of candidates that will never actually enter the intelligence community, whether because they are seduced by the higher salaries of the business / competitive intelligence sector, or when they find they fall into that percentage that cannot obtain a security clearance. The distortion caused by the inclusion of these individuals within the benchmark study renders any conclusions highly suspect.

It is long past time to return the MBTI to its place as a teaching tool to cultivate introspection, rather than the clumsy lever of management others have sought to make of it. We also hope that the intelligence studies academia will perhaps focus on those newer tools of workforce assessment and evaluation – such as the competencies model, or performance assessments based on the universal joint task list – which could yield viable results using a model student population.

We would also hope to see this research published in the community’s literature, or through the academy’s own press, in order that a more substantive debate might occur as to its merits, design and outcomes than speculation upon rumour would allow.

The sins of analytic methodologists

We are strong proponents of improving the rigour in intelligence analysis, and have frequently advocated for the better grounding of new analysts (and in-service training) in formal logic, epistemological theory, and structured technique. At the same time, however, we are deeply mindful of the continuing unmet needs for more creative and imaginative analysis, which moves further afield from the narrowly constructed questions of current uncertainties and into the realm of the unknown unknowns.

We are thus deeply troubled to observe a frightening trend in the work of some methodologists attempting to “solve” the problems of analytic rigour through new tradecraft practices. There is an increasingly common conceit that reliance on the analyst - subject to, cognitive bias, information overload, and human fallibility – can be engineered out of the process of doing intelligence. Instead, certain methodologists would substitute organizational structures, workflow re-organization, and the introduction of supposedly superior quantitative metrics in order to create a new standard for “answers”. The underlying thrust of these efforts is to reform intelligence activities towards a more “repeatable” process, often described by industrial or scientific metaphors such “foundry” or “lab”. These typically originate from the engineering and technical intelligence disciplines, and are usually directed as criticism of typical all source efforts – particularly those grounded in social science fields or qualitative methodology. Among the recent examples of these attempts at “standardizing” the process of analysis – with new emphasis on technical and quantitative approaches – is the 2006 DNI Galileo program paper “Predictive Network Centric Intelligence.”

The fundamental flaw in many of these methodologists' efforts is that they are essentially reductionist attempts to force the difficult and oft-times messy art of intelligence entirely into the narrow box of its scientific side. While there is a place for scientific approaches, particularly in the grounding and validation of assessment, the inherently creative, non-linear, and even non-rational elements of the profession can never be completely discarded. Most recent intelligence failures have occurred, not due to a lack of precision in judgment, but from a lack of imagination in identifying, describing, and forecasting the uncertain dynamics and emerging complexities of fast-changing accounts.

Most methodologists are very good at creating shiny and intricate machines for solving puzzles to produce answers of intelligence “fact” – often with great and noteworthy expenditure of time, effort, and resources. However, the community is challenged most today by mysteries, to which there may never be a single set of answers but rather only approximations and options, set against the backdrop of an ever evolving understanding of situations obscured by the fog of war, time/distance, culture, and levels of commitment.

It is also fascinating to note that many of the methodologists reject individual level approaches to solving difficult intelligence problems, preferring instead to focus on the creation of new systems and processes. It is our contention that the analyst will always remain the heart of intelligence insight, and the things that happen inside the human mind are non-replaceable - and frequently non-reproducible – elements that cannot be dissected and pinned neatly for methodological examination. The analysis “bench” will never be deep enough to support the creation of large structures of process in which analysts become interchangeable parts. Analytic tradecraft can provide best practices to help avoid certain fallacies and other mistakes, and can demonstrate cases in which valuable lessons can be learned, but it cannot by itself produce the innovative and imaginative prediction that will help decision-makers address the kinds of challenges they increasingly face.

It is for this reason that we continue to emphasize the importance of the cultivation of the analyst above all else. We would hope that the methodologists would at least consider pursuing research, which rather than seeking to radically restructure the forms of complex intelligence activity that have evolved under brutally Darwinian pressures, will instead support the pursuit of excellence by individuals facing hard challenges in the face of serious consequences. The analytic house neither needs nor particularly wants shiny new puzzle machines – but is desperate for additional tools and techniques which can be called upon to address mysteries in times of crisis, and when other sources of inspiration have run dry.

Evaluating judgment

One of the most difficult aspects of the art and science of intelligence is evaluating judgment. To weigh the accuracy of any given forecast requires review after the fact, and in this field there is rarely the closure that allows one to go back to the historical record and call a deterministic score. Doing so in any timely fashion is ever more suspect – after all, the definitive review of Pearl Harbor was not completed until more than 20 years later.

However imperfectly conducted, the task is still a necessary one – particularly when the lessons learned are vital to the conduct of the ongoing Long War. There are dangers in internalizing the wrong lessons, and in fixing too much any particular mental model. But there is equally danger in a lack of self-reflection.

The ongoing press of current operations too often robs us of the chances for such reflection. And what after action analysis does occur is far too often driven by politicization or by attempts at justification, neither of which are at helpful to the practice of the profession.

It is thus pleasant to find occasionally an author whose work on epistemological matters transcends particular political eras and the crisis of the day. Dr. Tetlock is one such writer and researcher, whose work is a must read for all intelligence professionals. He spoke recently at the Long Now Foundation, which has a podcast available (for those of us who don’t have pre-filled iPods handed to us for the commute), even if their summary does not quite reflect the true nature of the discussion (being more of a pitch than an abstract).

Among the more interesting points Dr. Tetlock makes is that good analytical judgment is directly at odds with expressions of confidence. While it is a common maxim as of late to attempt to reduce the caveats normally associated with the language of prediction, his research shows that this is a dangerous fallacy. Writing for intelligence is not Hemmingway’s fiction – complex concepts must be expressed in nuanced terms, and the analyst will do his consumers a disservice if the shades of gray within these issues in favour of a false certainty to ease communication.

He also cites an interesting point regarding the true manner in which the human mind considers estimative probabilities. While the recent trend has been for many researchers and academics to push to greater ranges of strict quantitative expression, the statistical importance of such subtle distinctions are lost in the verbal and quantitative minds of the consumers. There is a good reason why levels of probability and confidence are expressed as they are in the community, and those who would seek to impose change in favour of their own pet theories should be most cautious when doing so. (For one of the better explanations in the open literature, see the declassified key findings from the July 2007 National Intelligence Estimate, which include the section “What We Mean When We Say: An Explanation of Estimative Language.)

Mirror imaging of emerging threats

Private sector intelligence firms face a not inconsiderable challenge in marketing intelligence and security solutions to the corporate audience. Many multinational firms absolutely require robust intelligence support to tackle a wide range of transnational issues, many the very leading edge of emerging threats that will later become major interests for their counterparts in national intelligence organizations. The response within the corporate framework is often fragmented between various elements of the hierarchy, and between various functional areas and business units. The use of external vendors has been on the rise for at least a decade or more, but augmentation often occurs on an ad-hoc basis with little overall strategic perspective. It’s a devilishly difficult thing to get right – putting a new perspective to the arguments of those who complain about the growth in contracting in USG intelligence agencies. Thus, there are only a few variants on the “successful” formula which appeals to the corporate buyer of privatized intelligence and security services.

Recently, one of the information security vendors launched a new marketing campaign which is notable for an unusual display of creativity – not often a characteristic one associated with computer engineering. Hackistan is a fictional national adversary, complete with backstory, internal politics, and a national flag, that has become the vehicle used to pitch their product lines.

At first glance, it’s good for a laugh. And were it not for our personal experiences with particular flavours of Russian and Middle Eastern influenced threats, we might even fight it whimsical. However, the very resonance which makes the campaign successful from a branding perspective also gives us pause upon reflection.

It is not just the corporate sector which is seeking a more easily characterized adversary in a world of rapidly shifting amorphous threats. We have seen too many in even the best intel shops attempt to carve out their own version of branding for targets and issues. Sometimes, these are helpful mental models in the short term, creating better briefings and faster break in periods for new analysts. However, too often they are subtle traps for a whole host of cognitive biases, used to gloss over the difficult to answer uncertainties and the hard gaps. They allow mysteries to be recast into the familiar boundaries of yesterday’s puzzles, and offer the seductive but utterly false promise of understanding to which both good analysts and false prophets alike are inevitable drawn.

We have seen these crutches employed across accounts without regard to their utility – the recasting of the complex discussions of organizational theory around clandestine and covert threat groups into “terrorism, inc.”; or the attempts to simplify the shifting alliances and constant political interchanges between various warlords and chieftains through throw-away references to centuries’ old ethnic and tribal conflicts.

It takes real discipline to separate convenient metaphors from the necessary abstractions of analytical frameworks. It requires constant reminder that the map is not the territory, and that the familiar assumptions of our environments – be they political, military, social/cultural, or economic – may not serve us at all outside of our boundaries. And the future is just another abstract line of demarcation, constantly pressing in against the edge of the battlespace.

Combat stress and society’s (manufactured) reaction

Combat stress is a real thing. There are robust programs in place in DOD and elsewhere to help handle the effects of that stress on the human being, for war is a distinctly unnatural state of affairs and prolonged participation in it can do things to the health of one’s mind. Unconventional units, particularly those engaged in the delicate business of intelligence in support of counterinsurgency and counterterrorism, suffer combat stress more than most. So we, like many in our profession, are greatly cognizant of the hazard – and to a great extent, much of the community’s culture evolved to help place such stress in a larger context, and share it among those whom one trusts – especially those parts of the culture which have been so roundly criticized by those seeking political correctness and sterility.

The most important lesson we ever learned was that combat stress is driven primarily by one’s own reaction to society’s reaction regarding the stress events. The lesson came from the law enforcement realm, where the exposure to serious trauma and emotional tension was far more routine, and far more frequent, over the course of careers often spanning decades. These lessons have been validated time and again in the wars we have seen. It is not merely that one saw terrible things, or carried out difficult deeds during a period of time in a bad place. It is the cultural, political, and interpersonal context of what those things mean back in the World that can settle the restless nightmares or spur the haunting demons that lurk in a man’s mind.

There are outlets for this kind of stress, and means by which it may be handled. There are avenues available that those who need help may turn to. There is no shame in it.

Every man reacts differently to the stress. Physiological symptoms, psychological conditions, and triggering factors all vary. But there is no more sure means to convince a man that he is suffering than a relentless media campaign that tells him he should be. The propaganda in this case creates a reality. And this is a direct and grievous wound to those who should be offered instead support, care, and a greater context into which they might place the burdens of their days and thus see them lightened.

It is for these reasons that we view the recent Washington Post (and other media outlets’) attempts to stir up the ghosts of conflicts past to destroy support for the Long War as contemptible beyond measure. The media manufactured Vietnam-era PTSD coverage has proven over the careful examination of history to be driven by frauds and liars, who sought to cover their inadequacies in the stolen glories of others, and in failing sought to discredit entirely the idea of heroism, and bravery, and valour. (One of the best works documenting these false claims, and the media profiteering on the backs of such lies, was written by B.G. Burkett.) It is with this firmly in mind that we note the troubling similarities with current media accounts.

We view this most damnable of frauds as arising from an impulse that the Bard identified long ago in one of his more stirring military scenes:

And gentlemen in England now-a-bed
Shall think themselves accurs'd they were not here,
And hold their manhoods cheap whiles any speaks
That fought with us upon Saint Crispin's day.

Evaluating intelligence instructors

Economists often are responsible for some of the most interesting and unlikely of intersections with the intelligence studies field. The study of value, and how it is measured, has great importance when looking to establish the worth of activities which take place in non-traditional markets for which classic financial elements may not be the best set of data upon which to base metrics.

To this end, we are grateful for Marginal Revolution’s link to a study by a team of psychologists that attempts to grade teaching staff based on students’ rapid “thin slice” assessments.

Ambadi and Rosenthal. Half a minute: Predicting teacher evaluations from thin slices of nonverbal behavior and physical attractiveness. Journal of Personality and Social Psychology. Mar 1993.

The potential implications of this study are of interest not only to those that must manage the effective instruction and mentoring of the next generation of analysts and officers, but there are tantalizing suggestions that similar dynamics may be at work when finding a successful briefer. Given that most decision-makers tend to be more extroverted, and outcomes oriented, the tendency of these individuals to rely more heavily on rapid conclusions drawn from initial thin slice impressions weighed against their own knowledge and experiences, is likely to be even more pronounced than the average student.

We have long been interested in the implications of the psychological aspects of rapid cognition in intelligence analysis. (Malcom Gladwell’s work Blink provides one of the best and most accessible overviews of the subject, and the deeper body of literature on the subject is itself well worth exploration.) While we are strong supporters of structured analysis approaches and techniques, we recognize that in many cases the tradecraft currently taught for structured analysis may be woefully inadequate in specific situations. We have particularly observed the failures (at least of application if not of underlying methodology), of structured analysis by those in tactical and operational support roles, such as in watch centers or forwarded deployed elements – in no small part due to time constraints under overwhelming information volumes.

Successful analytical approaches in these environments focus on triage, linkage, narration, and perhaps most importantly the rare kind of projective imagination which is essential for warning and prediction. We strongly suspect that a better researched study of rapid cognition in intelligence applications would help to more accurately evaluate the specific strengths and situational weaknesses of the kind of day to day work that actually occurs in places where the formal structured analysis tradecraft has failed, taken out of the strategic and longer term production contexts in which it was first proven. Currently, proponents of structured analysis have attempted to dismiss alternative methods as “immersion” or “guessing”. Strict constructionists often define these other traditions so broadly as to encompass any technique other than a written matrix variant of ACH, due to the availability and emphasis of the literature attempting to copy Heuer’s excellent treatment of the subject.

Needless to say, this overly dogmatic approach is in our opinion unhelpful. We believe that while there are well founded conclusions which have emerged out of Heuer’s and his successors’ research, there are also hitherto yet un-quantified factors which contribute to the utility and validity of other approaches, and that further study is needed to identify and evaluate them in new contexts.

It is research such as cited above, as well as personal experiences in a range of very non-traditional and unconventional roles that strongly lends us to give credence to the possibility that there may “something different” with value to specific roles and functions within the intelligence community. To date, regrettably few academics in the intelligence studies field have shown any inclination towards such research, instead preferring the easier path of magisterial assertions that gloss over subtleties of craft in favor of strictures enforced with near religious intensity. And perhaps that alone is the most damning evaluation of most of those who claim to teach the art and science of intelligence after all.

We are hopeful, though, that as the intelligence studies field widens, and as new and more innovative minds enter the profession through non-traditional pathways – bringing with them their unique experiences and insights – that we will see new evolutionary progress in these areas.

The illusion of negative proof in incomplete operational demonstrations

We have in the past cited John Robb’s earlier works with interest, as it was clear he was attempting to piece together an understanding of the complex dynamics within modern sub-state and non-state networks involved in many of the key conflict areas around the globe. For these contributions to the literature we are indebted to him, even as we may productively argue with specific points.

We have been less enamored of his recent writings, however, but have been willing to leave the task of debating “multi” generation warfare (4GW, 5GW, et al) to those among the many excellent writers specializing in such military theory - who also not coincidentally offer a better turn of phrase on matters which we consider to very nearly border on combat theology, but for whose efforts we remain grateful.

However, Robb’s recent writings on “negative proof” to evaluate the effectiveness of current surge counterinsurgency operations in Baghdad demand a response. The logical fallacy of seeking to establish indicators for the operational and even strategic picture from single tactical incidents is bad enough. Although we believe in the strategic corporal, and have seen repeatedly the effects of major “turning point” incidents in Iraq (such as the bombing of the Askariya Mosque in Samarra), we have also seen far too many events designated by the pundits as the karmic keystone of the entire conflict – and just as quickly forgotten.

We are particularly concerned by Robb’s selection of recent events which form the basis of his “proof”. The enemy gets a vote in operations, and the success or failure of an effort is not judged on the actions the OPFOR takes in casting that vote, but rather are weighed in the net assessment of both sides’ effects. The strategic estimate is not merely a first draft of history, but the Choosing specific symbolic incidents such as the April 2007 complex attack on the Iraqi Parliament, or random protests (many of which consist of paid agitators), as meeting some threshold defined as “significant enough”, is deeply problematic. This is the essence of satisficing, and carries with it both the weight of cognitive bias, and also in no small measure elements of an analyst perhaps over-attached to his personal theory – with attending loss of objectivity.

The enemy understands well the non-kinetic effects of specific tactical actions, and chooses to utilize those that have information operations aspects without regard to the longer term efficacy of the options. One has but to witness the emergence of the red-on-red incidents between AQI and Sunni insurgents in Al Anbar to identify strong evidence of a mixed scorecard in the longer term assessment from what were seen as early “significant” events, such as the temporary seizing of built up areas.

Analysts should always be mindful of the necessity to rely upon rigorous analytic tradecraft, and in particular robust counter-deception practices, in order to prevent their work from being tainted by adversary IO themes – particularly when those themes are pursued through a pattern of kinetic operations rather than mere rhetoric.

The success or failure of current strategies has yet to be weighed. Perhaps the effects will be better measured in another Friedman unit… but it is more likely that the political debate will have moved the goalposts for the assessment by that point in time, and the then current discussion will be dominated instead by another transient issue that fails to reflect the grand strategic picture.

Closely related to this discussion is the troubling lack of awareness of enemy IO and political warfare operations and tactics, and the effects of these behaviors on analysis. These issues are not limited to any one pundit or thinker, but rather broadly impact many of those otherwise bright minds attempting to tackle difficult problems.

Fortunately, the academy has responded with at least one individual’s effort to remedy this otherwise widespread ignorance. Dr. Michael Waller has authored an excellent overview, building upon his earlier work countering Soviet and insurgent aktivny mera operations in previous conflicts. We are fortunate to count the gentleman among our influences, and highly recommend his instruction at the Institute of World Politics.

Public debate of intelligence issues

One of the more interesting evolution in the foreign policy and national security affairs communities that has been created by the interwebs is the increasing efforts by a number of academics and experts outside of the community to examine core, current intelligence issues in public debate and writings.

This is of course a quite understandable impulse, especially given the attention attracted by many of the serious issues of the day. Such efforts can bring much needed additional perspective - and even simple additional eyes on target - that may surface issues not previously considered and generate alternative analytical lines worth exploring. (The benefits of such an approach have been well outlined by our friend Mr. Tanji, in far more persuasive form than we could pen.)

However, the limits to such approaches are also worth exploring, particularly as analysis efforts stray into matters outside of the realms of single discipline OSINT or DOCEX; or when core analytical methodologies are initiated from a position of advocacy rather than dispassionate objectivity. Given the great uncertainties in understanding of many of the hard target issues, particularly given the nature of active measures of disinformation which often poison the well from which open sources are drawn, the development of inferences and defense of analytical positions is, as always, a quite challenging task. But this lot is accepted by all analysts, regardless of discipline.

And the constant, painful awareness of the limits of such understanding absent other sources of information not accessible in the public realm also comes with that territory. While the intelligence community may emphasize the need for additional accesses to hard targets, one must always recognize that the vast machinery of national technical means and other collection programs generates unique information that will only surface into the public domain long decades after the issue ceases to be current. Indeed, the examination of the historic intelligence issues of days past, with the aid of the properly declassified documents, has long been the staple of the academic intelligence studies world – and has been among the most valuable contributions the academic world has made to the field.

Some recent academic efforts however have increasingly trended towards what may be construed as very aggressive positions on certain issues, including directly challenging core intelligence community conclusions. Challenge is good – it provides a pressure within the marketplace of ideas that the community may sometimes be otherwise shielded from, although interagency coordination fights and red cell analytical lines are certainly their own form of competition and should not be blithely discounted. However, many of the recent challenges have been made using only partially declassified versions of core intelligence documents – or worse yet, leaked documents which may represent only one small portion of the available finished conclusions, let alone the supporting indicators for those conclusions. Such leaks, and leakers, are both reprehensible in that they represent not only an abandonment of the oaths of secrecy and loyalty sworn by those few privileged to be trusted with such issues; but also due to the typical political motivation and attending bias which has driven the selection and framing of the leaked documents. And whether based on leaked or only partially declassified documents, it is certain that the full range of available information developed by community sources and methods has not been revealed.

Among the current issues that have received such treatment have been debates over explosively formed penetrator (EFP) Improvised Explosive Devices (IEDs) [1], chlorine augmented payload Vehicle Borne Improvised Explosive Devices (VBIEDs) [2], Iranian WMD programs’ status [3], Chinese ASAT capabilities and intentions [4], North Korean nuclear programs [5], the scope of the dismantled Libyan CW program [6], and even the “strategic logic” of suicide terrorist attacks [7].

These debates have been interesting to follow, and one can respect the intellectual firepower that has been brought to bear by many learned individuals. Those who have put forth the effort to develop unique research sources, innovative alternative hypotheses, or technical analysis of difficult datasets, are to be commended in their efforts and encouraged to continue to bring new approaches and theories to the table.

We view these discussions with some growing unease, however, in that many commentators are increasingly straying from the open ended inquiry and exploration of difficult issues, conducted with the full realization that the levels of uncertainty inherent in the public discussion of such accounts will preclude any definitive statements. Rather, many experts are formulating positions – often heavily influenced by, if not outright grounded in, larger domestic political issues - and defending those positions with unfounded certainty or pure advocacy. This is unhelpful to the pursuit of truth, to say the least.

This should not be surprising, given the course of other public debates even in the most scientific and rigorous of academic disciplines. Apparently, there is something in the nature of the public discourse – be it personal reputation, cognitive biases, funding pressures, or something else entirely – that introduces subtle taint to the process. The intelligence community is by no means entirely immune to such factors, but it does benefit in decades of enshrined intent, and a solid corpus of analytic tradecraft and specific methodology designed to surface and defeat such intellectual contamination.

A most interesting related discussion in the context of global climate analysis can be found here - again from one of those most unlikely sources, a science fiction writer.

Mobile consumers

The increasing prevalence of real-time handheld mobile email solutions of the crackberry variety is certainly a well entrenched fact of life in the Beltway and other highly urban areas. The following item highlights interesting factors that impact decision-makers who receive information via such channels, versus more conventional communications methods (the political aspects of the specific situation aside, and frankly entirely irrelevant to our discussion).

Within selected organizations, the use of the crackberry to handle almost all material (at lower handling levels) is legendary. Just as community best practices changed to reflect the differing production requirements embodied in web-based dissemination, so to one would think changes are due in the face of these new changes. However, few shops have given any serious consideration to these issues, in no small part due to the fact that Blackberries and their ilk are “just another email option”.

Ironically, it is the homeland security organizations – particularly those at the state and local level – which are leading the way in changing both products and consumer expectations to embrace the mobile intelligence consumer. They face similar, if not sometimes more severe issues of security, handling, and control; but the lack of structure and technological support which has so plagued them in other areas for once appears to be working in their favour.

It remains to be seen what lessons might be learned from such dynamics elsewhere in the community. There is a serious gap of real and systematic research into the effects of new technologies on consumers, and the solutions intelligence producers are finding to help meet their clients’ needs in this new environment.