Wx-ing historical

We have addressed the recent fad towards addressing climate change as an intelligence issue several times over the course of this blog. We still remain convinced that while weather intelligence – Wx – will always remain a key factor in many accounts, climate change as a long term issue is simply beyond even the horizon that can be expected of the deepest of futures intelligence looks.

We are apparently in the minority in this view. We recently acquired a copy of an unclassified 1978 research paper from Central Intelligence Agency’s National Foreign Assessment Center (now republished by the University Press of the Pacific in 2005) which examined this very issue under the title “Relating Climate Change to its Effects”. For those younger analysts unfamiliar with the misty back history of old bureaucratic battles and therefore older acronyms, NFAC was the renamed Directorate of Intelligence (DI) under DCI Turner in 1977 – a designation which lasted only until reorganization under DCI Casey’s tenure in 1981.

We think perhaps this product might have best been buried with the old name. As a paper, it is almost entirely uninspiring – a mere 8 pages of substantive text, followed by hundreds of pages of tables and statistics that are the hard copy rendition of a contemporaneous data tape, mostly consisting of temperature and precipitation measurements assembled by a university contractor on behalf of USDA. These form the inputs to a simple climate model that was intended to provide for long term predictions of weather effects given specific outcomes, such as global cooling - a key concern of the day. (However, to their credit, the designers did examine the potential for global warming as well – which speaks well of the analytic rigour of the DI under any name, even if the paper is mute testament to just how badly scientific and technical experts can be at communicating with their readership through written intelligence products.) A speculative product such as this can be expected to offer no real conclusions – rather simply serving as a possible set of boundaries within the uncertainty space of future scenarios. However, it might well have made more explicit the effects it claimed to consider within the range of those scenarios.

But again, this serves to illustrate both the waste and the foolishness of attempting a futures intelligence estimate so far into the out years. We are not issued crystal balls when we are granted entry into the profession. It also serves to illustrate the perils of the arbitrary application of quantitative analysis as a fig leaf over unsustainable judgments. The model – no doubt painstakingly assembled and hard fought at the methodological level – is by its very nature the product of 1970’s era computer science. In the face of Moore’s Law, it is therefore over 20 generations obsolete.

Let us hope that this little musing upon the history of the account gives at least slight pause to modern practitioners seeking to enshrine climate change as a permanent account for long range intelligence analysis. At a point in time when supercomputational problems that take longer than a few months run time simply are not run at all until the next generation of architecture advances its inevitable order of magnitude or more, it is after all more than a bit presumptuous to assume that any community entity would be able to beat or even match the kind of big iron thrown at these problems in the civilian science world. It also very much begs the question of what better use such resources might be put to for other intelligence accounts – perhaps in the classic roles that the IC has always employed supercomputing resources: cryptanalysis, automated signal processing, or even exploring the new boundaries of potential offered by quantum intelligence.

If nothing else, this bit of history has also more firmly reinforced our opinion that climate change issues are a matter best left to the academics. Perhaps once the Long War has been won – and given the timescale we believe will be needed to accomplish this monumental, generational task – then the community’s attention can turn more to the matter once again. And if the current crop of speculative forecasts prove correct, at that point in time the issue may properly fall within the window of an actionable long rang estimate.

Obsolete intelligence professional skills

One of the newer of the memetic ideas sweeping through the blogsphere is the recent trend towards listing those obsolete skills of yesteryear which were once considered basic aspects of professional life, yet now are dead in the face of technological and social change. There is even a new wiki set up to catalogue them.

We have briefly been considering those which apply to the intelligence community. Of course, in a profession as conservative as ours, many of the skills which were initially considered for our own list are actually still frequently taught within the community on the off chance they might be employed at some point. And given the far flung enterprise of the Long War, many once obsolete skills are actually being used out in the field, whether out of simple expediency or out of a lack of any alternatives.

Thus inclusion of a skill as obsolete on our list does not necessarily mean it is without any further value. However, we might venture to say that it might not be the first skill we seek to inculcate in basic professional coursework, despite many models which currently seek to teach new analysts as if they too were undergoing the chronological developments of the profession’s technological support.

• Hand stenciling network analysis link charts
• Annotating developed imagery prints by hand
• Developing handheld imagery in a darkroom
• Loading and unloading imagery satellite film canisters
• Maintaining card indexes
• Carbon paper copying
• Conducting pigeon reconnaissance operations
• Transmitting morse code
• Operating hand crank cipher machines
• Memorizing poem codes
• Sending coded postal letters
• Telexing cables
• Changing the road signs outside of headquarters building
• Inking overhead transparencies
• Delivering pneumatic tube memos

Submissions from our readers are welcome. Several that might have made our list, such as operating reconnaissance balloons, servicing dead drops, or donkey riding, have enjoyed a recent renaissance due to new developments in the contemporary environment. Such items are still welcome as submissions in their own right.

Imaginary constellations meet visionary capabilities

We rarely comment on ongoing matters in the intelligence world due to the natural restrictions which accompany a professional’s responsibilities. However, the very public intercept of USA193 does touch upon an issue that we have previously discussed in these pages – the disconnect between what assets those planning for the future intelligence community thought they would have, and those that exist in an IC in which we have gone to war with what we actually have.

The shot itself has been extensively discussed by others, including In From the Cold, Arms Control Wonk, Ares, Danger Room, and many of the more mainstream blogs. We will not rehash these public details.

However in this particular instance, we find the contrast between the notional pieces on the chess board (and the detritus they left behind when the model came tumbling down) quite striking against the very effective real world capabilities offered by one of the most condemned warfighting concepts of recent history - hit to kill. These differences are also at the heart of the classic tensions between intelligence and operators. But it also demonstrates the value of innovative vision – and the need for persistent effort towards what may seem an unachievable goal. The operation also shows the adaptability of those capabilities towards missions that may never before have been conceived – particularly when the tools to execute them were first on the drafting boards.

The debates of the past few decades across all of the systems involved, and last night’s operation itself, also brings to mind the Bill Whittle’s now seminal essay on Tribes. The defense space community gave the public an unprecedented glimpse of world of the Gray at the high frontier. The press briefing (found here) is as specific a discussion as ever has occurred regarding a once entirely classified area. One has to remember that only a few short years ago, even the name of the National Reconnaissance Office was not publicly acknowledged – let alone the kind of systems of systems approach evident in this operation. But the briefing also displays strong analytic tradecraft, and an excellent use of estimative language to communicate intelligence and operational information in a transparent manner. It was indeed an impressive display, and is well worth studying for those that are in the business of facing tough talks.

Altogether, this was a unique mission, and those involved will be able to look back on the operation with pride. They have most deservedly earned the drinks being poured last night and this morning. Space control has a new face. Let us hope that this will also be the symbol that inspires future generations of capabilities on the intelligence side of the house, and the desire to avoid any other holes in the imagined sky.

On analytic environments

It has been a while since we have revisited this subject. We are still strong believers in the adage that if one stares long enough into the cubicle, then the abyss stares back. However, long gone are the halcyon days of an IC influenced by the dot com era, seeking to implement revolutionary new space designs for a more creative atmosphere. Now, we face constant compression, and the ad hoc creation of new kinds of environments in the far flung realms of our forward deployed edge, as well as the unique spaces of the watch and fusion centers that now proliferate throughout the community.

But it is occasionally worth reflecting on what might be, if one were to challenge the dominant archetype of the current analytic environment. New spaces are being built all the time, and the further one goes from the Beltway, the more potential one finds for innovation – especially in the face of decentralization pressures.

We are not looking for something so radical as to be out of place even in modern corporate culture. Certainly nothing like the brooding industrial era estates one finds out in the wilds of “other” Virginia that might be readily re-purposed to the cause, but would remind one of a nearly HP Lovecraft atmosphere better fitted to the home of the fictional Laundry (or perhaps more appropriately, its American Black Chamber counterpart.)

Rather, we look to the best in class commercial entrepreneurs whose primary business is that of the mind. We have written many times before about the approaches taken by Google, and think enough has been said for that comparison. We would this time around seek to highlight the new spaces created for Microsoft’s Research division, also as iconic an institution of thought as any in modern America. We are fortunate that the roving blogger Robert Scoble has profiled this unique environment in a recent photo series.

We think there are lessons in these designs which can be distilled for the new IC. We are certain that given the option, many of the best and brightest would vote with their feet in favour of such environments - should they ever become available in an enlightened organization.

An alternative film for intelligence analysis exercises

We have been surprised at the level of discussion sparked by our previous consideration of the classic 12 Angry Men and other films, for teaching the fundamentals of intelligence analysis and writing to entry level candidates. Apparently, the technique is more widespread and popular among instructors than we knew. Frankly, we have found that it is something possible only in professional in service sessions, where there is usually enough contact time to permit such hands-on practical work. Typical academic environments rarely afford sufficient engagement to lose the several hours involved, and we have found that most students assigned to view the same film on their own will tend to present remarkably similar work product (although the assignment tends to be very popular with students, particularly prior to weekends and in coed classes.)

We have thus given the matter some consideration, seeking a better replacement for the now classic Joint Military Intelligence Training Center exercise which utilized the film The Hunt for Red October (briefly mentioned in our earlier post on the topic.) That film was an excellent choice for the time, insofar as the limitations of Hollywood typically permit – and was even used as a teaching aid at the submarine school in Groton for a time after its release (though albeit moreso for its counterexamples). Its utility was no doubt derived from the unique historical basis for Clancy’s original book – a composite of several real world cases of great strategic importance during the Cold War.

The purpose of the film exercise was to provide the analyst student with a chance to practically apply basic tradecraft dealing with ambiguity, and to create a collaborative analytical product using simplified unclassified material. Each student would be assigned to a smaller group, which would then be tasked with a specific intelligence component to focus on. The students would then seek to answer a key question regarding the factors which influenced the defection of the fictional Russian submarine captain in light of these larger strategic issues within the Cold War. The film material itself was treated as authoritative narrative – requiring a degree of suspension of disbelief, but not terribly so if the students were unfamiliar with actual undersea warfare. This typically led to some excellent discussions and more than a few unique analytic outcomes.

Replicating this exercise in a more modern context has proven to be no easy task. There simply have not been films which encapsulate the unique factors which made the Red October exercise such a good choice. But as much as we liked the case, its value is limited for students who will be engaged in the Long War for some time to come.

We considered – and rejected – quite a few other options. The 1996 film The Peacemaker was one possibility, but for most students the Balkans conflict is as remote as its World War I antecedents, and WMD terrorism and homeland security issues are now viewed through a far different lens in the post 9/11 world. The 2005 film The Great Raid could offer potential, but was a more limited tactical scenario in a far less ambiguous information environment, from which actual historical materials would be far better suited as a source of instruction. We briefly contemplated the film Spy Games, but it is far more suited for a history of intelligence class than an analysis course. Syriana too came under consideration, particularly given the involvement of a former case officer in its scripting, but the explicit politicalization of the film also ruled it out. The drug film Traffic was rejected for similar reasons. Most of the other contemporary drama or action films can be dismissed out of hand, being little more than flights of fancy – something that removed films such as Swordfish, the le Carre works, and all of the Bond pieces from our list.

This left us with few options. Thankfully, our dilemma appears to be solved – for the time being – by the 2007 release of The Kingdom. It is unsurprising the film’s early releases were trialed in the greater Washington DC metro area, and that a high number of community professionals were among those early audiences. While the work suffers from the usual Hollywood inaccuracies and the insufferable modern politicization, it does present a narrowly scoped case which is itself a composite mélange of historical incidents of ongoing relevance. While we hate to be seen promoting the Bureau – particularly through the fantasist version of that organization presented in the film – if one ignores those aspects, there is value in the sense of realism otherwise conveyed across the piece through a good application of the director’s art.

Again, the key to turning the film into a good analytic exercise – rather than just a several hour long break from lecture – is to encourage deeper discussion of the underlying factors that led to, and would result from, the incidents depicted. The students should be able to pull out a number of specific points that can be summarized and expanded with additional open source research into unique finished analytic papers. The film offers a variety of these springboards – from the tactical aspects of attack TTP, to terrorist propaganda operations, to the role of re-integration programs for former terrorist prisoners, to profiles of host nation CT capabilities, or to the issues of radicalization within specific industries, geographies, or societal segments. The instructor may need to assist the students in settling upon these aspects during post-screening discussions. This is less a group product, although a collaborative framework can be created in which individual papers support a larger work, especially using a wiki production environment.

Despite finding what we feel is a good solution for the time being, we will continue to seek out alternatives for use in other areas of the contemporary intelligence domain beyond the CT sphere – and will of course welcome any suggestions (along with reasoning in defense of the choice) that our wider audience might contribute for general circulation.

Evidence inferred

In the rush to focus on new analytical techniques and methodologies, particularly the more complex and arbitrarily numeric variants, we think that the fundamental aspects of inferential reasoning are too often overlooked. This is by no means the stuff of such modern vogue as complicated algorithms or cutting edge research into cognitive developments. Nor is it the drumbeat of “critical thinking” applied absent real problems or even properly constructed examples.

Rather, this is the meat of what separates the analytic mind from that of the mere reporter. The problem occurs not only when building new analysts, who are too frequently shorted on the fundamentals which are vital in cultivating this distinction, but also when developing analysts’ tradecraft at the journeyman level.

Part of the reason why inference is so often given such a short shrift is that there is rarely new material of interest in circulation on the topic, when one can easily find a few hundred other resources for just about any new “hot” topic. But then again, one supposes it is hard to muster funding for a proposal described as an attempt to revisit and extent the work of a lawyer from the late 19th century.

It is for this reason we were delighted that the folks at University College London have put together an excellent cadre of interdisciplinary researchers to tackle the problems of evidence and inference, including the esteemed Professor David Schum (who first brought the phrase into common usage within the intelligence profession). The Evidence Science group is clearly seeking to forge new ground from what others have long ignored as well trod paths. Their publication set is indeed well worth reviewing, as it covers a wide range of topics central to the profession of intelligence: the persistence of discredited evidence, exploration of belief formation, and the use of rhetoric and argumentation, among many others. While only a few of the pieces focus explicitly on the intelligence domain, these exist within a selection of interdisciplinary material which also can provide its own value to the judicious reader. (It is for this reason that we can recommend the site despite the number of lawyering and quantitative analysis publications).

We think this might be just the thing for some of those in the Beltway contemplating a long weekend away from it all and looking for some not-so light reading in the wider literature.

Another for the wall of dead terrorists

Once upon a time at a particular institution, new analysts were assigned to a simulation course which involved a model joint terrorism task force. Over the course of time, they built up quite a collection of handheld imagery of various prominent targets, many of which came out of historical case studies. This collection was for a time assembled into an infamous “wall”, in which the sole factor for inclusion was that the target has been serviced and resolved.

Via MESH –Middle East Strategy at Harvard – we learn that the infamous Lebanese Hezbollah leadership figure Imad Mughniyah is dead in a classic Beirut style car bomb. His brother was previously killed in the same fashion.

This one has been a long time coming. We are sure that many of those former junior analysts – now having progressed for quite a number of years in the field, and with many a now forgotten face having been posted to that wall since – may be quietly ordering a round as a toast to a small measure of victory. It does not matter, in that moment, whether this was merely red on red violence, or if some unknown covert action element of the international great game achieved the decisive checkmate. It only matters that the faces now change, and the benefits of Mughniyah’s long operational experience has been denied to the terrorist adversary.

In good Buenos Aires fashion, we think our drinks for celebration shall be the Bellini – perhaps accompanied by a fine bife de lomo.

The use of leaked documents in intelligence studies education

This issue has been weighing upon us for some time, and has sparked perhaps the most violent debate of any subject within a field already crowded with passionate viewpoints. It takes on new prominence this week with the actions of another intelligence studies professor (names are omitted to protect the guilty) at one of the more prominent institutions out there.

We are unabashed supporters of the use of declassified intelligence documents – including finished intelligence papers, raw cables and other message traffic, and any imagery that might be available. These are almost without exception historical in nature, and thus we also advocate the use of unclassified notional intelligence documents produced in the model of current approaches (differing only in those areas that classification requirements dictate). We admit that the latter requires a lot of hard work – both in finding the unclassified or declassified examples from which to build templates, as well as creating the notional products that the students will rely upon in class or in an exercise. Many academic programs simply forgo this altogether for this reason – understandably so, but frankly in our opinion to the detriment of students that need exposure to “real world” intelligence in a form that may be properly used in a classroom.

However, the use of leaked classified documents in education is another matter entirely. Too many prominent names in intelligence studies publishing pad out their books with leaked documents, many of which can be said to be exceptionally damaging to United States interests in the subject under discussion. We have over the years grudgingly assigned these texts, as the better authors still offer some value to students despite the damnable offense of perpetuating leaks. What is most unfortunate in these cases is that those authors – by virtue of unique analysis or concise presentation of complex topics – would be entirely compelling without the leaks, yet apparently do not have the confidence to stand on their own, or the intellectual integrity to present their own work unaided by stolen secrets.

Such matters have long troubled the field – and frankly, have done much damage to the establishment of a respected intelligence studies academia that interacts with its professional counterparts in a mutually beneficial fashion, instead of through parasitic and self-serving profiteering.

There is however a more disturbing recent trend, one abetted by the evolving issues that come hand in hand with widespread electronic dissemination of intelligence products, and the inevitable friction that occurs when attempting to cope with the proliferation of classified networks and channels under wartime conditions. This new issue is the unprecedented availability of still classified documents (and other media) in their original form; leaked from improper handling - or worse yet, deliberate disclosure - onto the public Internet. These are becoming distressingly common enough that there are even now sites dedicated to the propagation of such leaks.

Without commenting on any specific incident, it is understandable that some civilian academics might see these are rare opportunities to provide a window into current intelligence practices for their students. It is also entirely likely that the “cool” factor may have overwhelmed good judgment when dealing with these cases. But we are exceptionally concerned that these classified materials not be routinely incorporated into unclassified academic instruction. Nothing will do more damage to the discipline as quickly as such an outcome.

First, among those students in many unclassified classrooms there are those that hold current clearances or other professional affiliations that impose a proactive and affirmative burden on the individual to report the improper handling of classified materials. It is unconscionable for an instructor to impose through their own deliberate actions this burden of time, paperwork, and ethical dilemma on a professional student.

Secondly, for those students that do not hold current clearances, many will one day face the polygraph process – and the discussion of a half remembered document from a long ago professor is not the most productive way to encounter the less than tender mercies of that process. As it is, too small a percentage of those students will successfully pass vetting; the intelligence studies academia does not need to be encouraging additional obstacles that will further negatively impact those numbers.

Lastly, we fear the creation of perverse incentives for future leaks should this practice become more widespread. We could easily see such pressures being placed entirely inappropriately on serving professionals who are alumni of major intelligence studies programs by their former instructors, or anonymous leaks occurring at the end of a professional’s tour in anticipation of a future academic posting. We cannot condone any activities that would potentially create any similar incentives – especially when such pressures might well result in the end of meaningful professional and academic collaboration partnerships in the intelligence studies field.

We know the difficulties in crafting an intelligence curriculum to be taught a the unclassified level, but have long felt strongly that to do so forces a focus on the fundamentals of tradecraft unhindered by the restraints of specific organizational niches. While there are many things that simply cannot be taught at the lowest levels, most are frankly more appropriate to a professional in service training and education program as opposed to the outside academic environment in the first place.

It is in part due to these actions that many community professionals entirely discount the role of outside academics – and we fear that with each passing incident, this perception becomes harder to fight. Given the behavior of some academics, it is a perception that may not even be wrong. We recall one particular foreign born instructor who, prior to his dismissal with prejudice from a particularly prominent program, had set out to deliberately acquire as many leaked materials as he could lay hands upon. This created serious difficulties for other academics and students in the program – many of whom were employed in consulting capacities for various official institutions. This is an example that should never have been allowed to be repeated – and current incidents are a slippery slope on that road.

Let us be clear: we at Kent’s Imperative condemn leaks in all forms, and those that would find benefit from them, in the strongest possible terms. Each academic institution which hosts an intelligence studies program should address this issue through internal policy – preferably tied to its academic code, which should consider the improper use of classified information as damnable as the kindred crime of plagiarism. If there is any role for the International Association For Intelligence Education in the promulgation of best practices throughout the field, it is in such matters.

Revisiting analytic rigour

The research currently being done at Ohio State University into the problems of intelligence analysis – including information overload, cognitive processes, and other aspects of the methodology – has from time to time caught our interest. Among the more interesting of these items now in circulation is an excellent lecture that we most highly recommend to our readers, recorded last year during the too often overlooked Google Talks series. The discussion focuses on the evaluation of analytic rigor, and means by which analysis may be strengthened.

We particularly favour the philosophy that Dr. Woods presents, which seeks to avoid dictating a single best methodology or process. We are more than willing to listen to the methodologists, but too often we find a dictatorial approach significantly at odds with the realities of line analysis. We think that the observed case study technique used in the Ohio State team’s research – something too infrequently done by many academics – is key to the validity of their findings. One cannot discuss analytic ideals without involving those who are actually involved in applying tradecraft to real problems. It is also not enough to conduct such research in artificial environments within student populations – real line analysis is too different.

We certainly cannot agree with the apparent off-hand condemnation of “folk” psychology of intelligence analysis – clearly aimed at taking on Heuer’s “bible”. While we think that there is a clear role for the methodologists and their research into strengthen analytic tradecraft, there is also a very real need for interdisciplinary adaptation from other areas of social science, as well as the kind of internal discussions that make up a key part of the maintenance of those oft-criticized, but entirely vital, guilds that are the backbone of the community.

We do however find several key concepts of great interest that deserve wider attention, including the concept of the Supervisor’s Dilemma – the balance of customer outcomes requirements and analytic resource opportunity costs against the relative depth of analytic rigour. We also find the study techniques themselves of interest, especially the concept of elicitation through critique – something we feel will likely have a far greater applicability in capturing the kind of intergenerational knowledge that the community is in danger of losing. We see the technique as one means of making more formal – and scalable - some of the kinds of subtle interactions that characterized the experiences of apprentice and journeyman analysts under the mentorship of a master.

We also find great merit in the good professor’s comments regarding the overconfidence of new analysts, and the satisificing biases that result. We definitely have observed a level of arrogance in too many new hires – and especially those coming out of the intelligence studies programs. The first lesson that an analyst student should learn is the fear of God – and of their own error. Too many programs of instruction are not affording the student the chance to learn that fear from the visceral experiences of their own mistakes, and to take away from the experience a humility that will cause them to productively question their future work toward its improvement. Such experiences are far better gained when the consequences are not fatal, in line with the lessons taught by Red Flag.

There is much food for thought in this lecture, as well as the contributions to the literature that the Ohio State program has generated. We will no doubt have further commentary on the subject in the near future.

123 Meme, with variation

Thanks to Mountainrunner, we have been tagged with one of these random interweb memes that seem to us to be a deliberate attempt to spark a convulsive degree of self-consciousness within the restless stirring of electrons that is the Parallel Universe. (And given the gentleman’s interest in all things UxV, one cannot discount his role as agent provocateur in welcoming the new robot overlords – after all, he is the individual that inserted reference to Cylons into a serious DOD briefing.)

For those that may have been thus far spared exposure to this particular viral idea

1. Pick up the nearest book (of at least 123 pages)
2. Open the book to page 123
3. Find the fifth sentence
4. Post the next three sentences
5. Tag five people

Given that our group format by its very structure creates a different response to this question for each participant, we found ourselves internally cross-tagged. After some discussion, we also decided to introduce an additional element of randomness in the passages selected – within a moderate degree of ambiguity near to the canonical 123 meme segment – just to stir the echoes a bit more (and no doubt as further evidence of our contrarian nature). After all, intelligence professionals should not become accustomed to too high an artificial measure of certainty in anything.

Thus we offer a few of the more notable passages from the various and sundry texts offered by our contributors:

Ransom, Harry Rowe. Central Intelligence and National Security. Harvard University Press. 1958.

“’New concepts’ earlier mentioned in State’s intelligence organization are partly the result of the influx of Foreign Service officers into intelligence – ‘Wristonization’ – and partly the result of a redefinition of intelligence requirements of the Department. Effort is being made to assure that the intelligence produced is attuned to the real needs of State’s policy makers and operators. Through various administrative devices closer daily operational contact exists between officials responsible for making and implanting policy and those supplying intelligence.”

Earley, Pete. Comrade J. GP Putnam’s Sons. 2007.

“Having tasted the reforms sparked by perestroika and glasnost, Soviet legislators were not willing to turn back the clock. This left the KGB chairman and his cronies with only one option. If they wanted to stop the Union Treaty, they had to remove Gorbachev with military force before the August treaty was signed.”

Sageman, Marc. Leaderless Jihad. University of Pennsylvania Press. 2008.

“Unlike traditional terrorist organizations that have physical sites and more territorial ambitions, there is no incentive for a leaderless virtual social movement to moderate or evolve beyond terrorism. Because there is no formal organization, with assets, sunk costs, physical commitments, or other stabilizing elements, participants who become more moderate in their views simply leave the forum and move on, or are banished from the forum by the webmaster. But their legacy lives on; their previous commitments and activities (writings, videos, and terrorist operations) are still archived in the forums and could continue to inspire new generations of dreamers to capture the glory that had inspired the old stalwarts in the first place.”

Anderson, Terence; Schum, David, and Twining, William. Analysis of Evidence, 2nd edition. Cambridge University Press. 2005.

“The logic is simple; the complexity lies in the materials to be analyzed and in identifying the relationships between the propositions in an extensive argument based on a mass of conflicting evidence. The logic is binary: every relevant proposition either tends to support or tends to negate a single hypothesis or conclusion (the ultimate probandum). The technique is dialectical: the aim of the chart-maker should be to construct the most cogent possible argument for and against the ultimate conclusion and to relate the opposing arguments within a single coherent structure.


But we do also wish to tag a few others, for we are far more interested in thoughts which originate outside of our little skunkworks. Thus:

• Deborah Osbourne, of Analyst’s Corner
• Jesser Wilson, should he resurface
• August Jackson, of SCIP
• Jeff Dexter, at the Ghost of Patrick Henry
• and Dr. Waller at Political Warfare

Revisiting Twelve Angry Men and legalism in intelligence analysis

For a number of years, the classic black and white film Twelve Angry Men has been a frequent teaching aid in introductory analysis courses dealing with the basics of evidence and argumentation. The conventional use of the film is to provide an accessible means of deconstructing a fictionalized scenario for students with little prior experience with formal debate. Given the current decline in public education, this helps remedy a basic skills deficit that is unfortunately and increasingly all too common.

However, it is with interest that we observe the controversy that has erupted once again over this fifty-one year old movie. The criticisms that have been leveled against the underlying premise of the film deserve some additional consideration – not the least of which because they point out the serious problems in applying much of what is taught as legal logic to the unique problems of the intelligence domain.

The Spectator’s argument surfaces one of the reactions common among many students, but in a far more articulate fashion than any entry level professional might be expected to voice. In essence, this criticism is based on the need to focus on the external worldview, rather than the tactical maneuvering in the courtroom that such kinds of arguments inevitably devolve towards. It is a quite valid point, and among the reasons that we have long decried the trends towards creeping legalism that have lately come to dominate intelligence work. The bulk of a lawyer’s litigative activities – and therefore a disproportionate degree of their education and professional experiences – are dictated by entirely tactical considerations that apply nowhere else but within the limited framework of the legal system. Too often this is easily forgotten, to the detriment of the strategic picture – and the accuracy and veracity of analysis. We have written on these problems before, but to be frank we had rarely considered the myriad of ways in which – by borrowing from the older legal profession’s traditions in teaching basic logic and rhetoric – the intelligence academia may continue to contribute to these unhelpful trends of cognitive bias. Among these, of course, are the kinds of ludic fallacy identified by Nassim Nicholas Taleb.

The folks at Volokh Conspiracy take a different tack, arguing for greater consideration of interdependencies within factors under examination in the film’s fictional trial. This is also a very useful approach for discussions with students, many who likewise tend to view evidence in isolation. It is particularly appropriate when covering structured analysis techniques such as ACH – and one that rarely touched upon, if only due to the frequently too shallow examples offered to illustrate the methodology, which are unable to support a more robust discussion.

Westminster Wisdom rises in defense of the film, and illustrates the more important but also often overlooked value to the piece in the intelligence studies classroom – the discussion of uncertainty. The intelligence professional will always work within a framework of ambiguity, doubt, and frequently, deception. However, the role of intelligence is not merely to reach a lower standard of proof than that used in a criminal trial (or even the lesser civil threshold), as is commonly taught (and in particular, a tenant of faith within law enforcement intelligence). Rather, intelligence’s purpose is to provide accurate insights despite such uncertainty; and where absolute accuracy is not possible, to bound the space of uncertainties for the policymaker in a manner that supports informed decision-making.

In light of all of the foregoing, we continue to search for good alternatives to the film for use in the classroom. Our search is also driven by the simple fact that to the Millennial generation, the black and white format is very nearly entirely alien. It creates such a visceral negative reaction that the first ten to twenty minutes of the film are simply an orientation to the unfamiliar environment. The pace of the thing is also glacial by modern standards, and particularly so to minds attuned to rapid multi-tasking and immersive information environments. While one can make all the arguments one likes about the need for sustained single focus attention, their native preferences are indeed more suited for the kind of world in which they operate as intelligence professionals than the Industrial era conventions that black and white film represents.

The single set format, and the emphasis on argumentation, has however made it exceptionally difficult to find a substitute. Our best – but admittedly still imperfect – alternative has lately been the 1999 film Deterrence, which offered a President’s decision-making process in a nuclear crisis while snowed in at a small diner. Unfortunately, the film’s scenario is constructed around a fictionalized Iraqi dictatorship – which in the modern politicized climate often steers debate too far astray of the real purpose and into the debate over Operation Iraqi Freedom. It also pre-supposes a certain level of student knowledge regarding nuclear warfighting and mutually assured destruction strategy: something not always guaranteed in the post-Cold War cohort. This sometimes makes for quite interesting discussions, to say the least. These are the same problems that incidentally also led us to abandon using the old JMITC exercise that relied on the film version of Hunt for Red October as a notional scenario from which the students would develop practice analytical pieces. While, as a friend recently reminded us, that particular film ages very well as such things go, we recognize that writing intelligence on Soviet era ballistic missile submarines is an anachronism to which few students will respond well –and one that does not serve their real and current professional needs.

All in all, teaching intelligence with films – for as engaging as the technique might be for students bored with lecture – remains a difficult proposition. We hope that in time the development of newer tools for digital animation – and the kinds of interactive scenarios that new gaming engines permit – will render the question entirely moot. But the cultivation of young professional minds remains a terribly stubborn business, and one that is not frequently improved by new technologies. We do wish to see such improvements become effective, and preferably in the near future. But given that the same promises have been made since around the time black and white films first graced the big screen, we remain skeptical.


h/t Overlawyered

Delving deeper into prediction markets

Michael Abramowicz of George Washington University has been guest blogging at Volokh Conspiracy for a short time now, and he has given us much food for thought on the topic of prediction markets. There is easily enough material for an entire book, and unsurprisingly, he has written one (that is now on our must read stack); as well as his own blog site. The Volokh post series has been:

• An Intro to Prediction Markets and the Liquidity Problem

• Prediction Markets vs. Conventional Wisdom

• A Quick-and-Dirty Empirical Defense of Prediction Markets

• Manipulation of Prediction Markets

• Deliberating with Prediction Markets

• Predicting Decisions and Their Effects

• Normative Prediction Markets

• Why Normative Markets?

• Nineteenth-century prediction markets

We will no doubt have more to say on the topic ourselves in good order. However, it is a subject that deserves deeper reflection, especially given our acknowledged skepticism of such efforts – and our general distaste for attempts seeking to create artificial numeric precision.

Nonetheless, we are quite grateful to the author for his work, which offers a unique contribution to the literature in an area of great interest to the IC. Whatever one may think of the technique, it is worth exploring with the same rigour as any new methodology.