Not an Intelligence Estimate

We have been aware for some time of the recent academic exercise by Mercyhurst College’s Intelligence Studies program in utilizing a wiki-based production process for analytic experimentation which resulted in the production of a model National Intelligence Estimate on infectious disease issues. It has now apparently gone public under the auspices of the National Intelligence Council.

We were initially very interested in the discussion surrounding this effort, as it seemed on the surface quite innovative, and touched upon many of the key areas we have focused on here at Kent’s Imperative – including the use of new technologies to support analysis, and the integration of private sector and academic expertise into community efforts. However, we had reserved judgment and comment until the final form was published.

We find ourselves now troubled by the results we have seen. While we still support greatly the concepts of distributed, virtual production utilizing new collaborative technologies to blend academic and IC professional expertise into a single cohesive product, we are greatly disappointed by the initial run of the experiment. We suspect strongly that those without substantive community experience may have played a role in the divergence from what seemed initially a good project into something that displays remarkable deficiencies in tradecraft and judgment – not to mention a depressingly shallow treatment of the subject under consideration.

We may offer a more substantive critique of the product in the future, but our initial coordination notes would highlight the following, both methodological and substantive:

• Nation-state level focus on what are essentially transnational and globalized problems

• Lack of any apparent utilization of futures intelligence methodologies (such as scenario projection, horizon scanning, Delphic analysis, driving forces / event mapping)

• Excessive confidence in statements based on incomplete information or future uncertainty

• Introduction of arbitrary quantitative metrics into an essentially qualitative expression of analytic confidence, rather than utilizing intelligence community normative forms

• Too frequent focus on current conditions as opposed to the supposed 10-15 year target horizon of the product

• Layers of embedded analytic judgments expressed as factual points in “editorial” style

• Lack of any serious examination of Non-Governmental Organization and other private sector roles, currently or in the future timeframe of the study

Some might say we are being too harsh on what are after all a group of students. We do not direct our criticism at the students themselves – they clearly made an effort to address a very difficult task normally given to professionals with many years more experience (and a much greater familiarity with DI writing style); and acquitted themselves as individuals well. Our disappointment lies solely in seeing an unprecedented opportunity to a great extent squandered in the production of a glossy rag rather than something of true and enduring value. We continue to believe that even these most junior analyst / students are capable of much more – and that the concept has the potential to offer unique value to the intelligence community.

Still, we are glad that something came of the effort, if only to observe the process from afar. We would like to second Haft of the Spear’s call for the introduction of new schools, and new groups of academics for new experiments on new targets - that may perhaps have a better go of it, particularly given greater involvement of IC professionals in the kind of blended approach we have been advocating for some time now.