Evaluating intelligence instructors

Economists often are responsible for some of the most interesting and unlikely of intersections with the intelligence studies field. The study of value, and how it is measured, has great importance when looking to establish the worth of activities which take place in non-traditional markets for which classic financial elements may not be the best set of data upon which to base metrics.

To this end, we are grateful for Marginal Revolution’s link to a study by a team of psychologists that attempts to grade teaching staff based on students’ rapid “thin slice” assessments.

Ambadi and Rosenthal. Half a minute: Predicting teacher evaluations from thin slices of nonverbal behavior and physical attractiveness. Journal of Personality and Social Psychology. Mar 1993.

The potential implications of this study are of interest not only to those that must manage the effective instruction and mentoring of the next generation of analysts and officers, but there are tantalizing suggestions that similar dynamics may be at work when finding a successful briefer. Given that most decision-makers tend to be more extroverted, and outcomes oriented, the tendency of these individuals to rely more heavily on rapid conclusions drawn from initial thin slice impressions weighed against their own knowledge and experiences, is likely to be even more pronounced than the average student.

We have long been interested in the implications of the psychological aspects of rapid cognition in intelligence analysis. (Malcom Gladwell’s work Blink provides one of the best and most accessible overviews of the subject, and the deeper body of literature on the subject is itself well worth exploration.) While we are strong supporters of structured analysis approaches and techniques, we recognize that in many cases the tradecraft currently taught for structured analysis may be woefully inadequate in specific situations. We have particularly observed the failures (at least of application if not of underlying methodology), of structured analysis by those in tactical and operational support roles, such as in watch centers or forwarded deployed elements – in no small part due to time constraints under overwhelming information volumes.

Successful analytical approaches in these environments focus on triage, linkage, narration, and perhaps most importantly the rare kind of projective imagination which is essential for warning and prediction. We strongly suspect that a better researched study of rapid cognition in intelligence applications would help to more accurately evaluate the specific strengths and situational weaknesses of the kind of day to day work that actually occurs in places where the formal structured analysis tradecraft has failed, taken out of the strategic and longer term production contexts in which it was first proven. Currently, proponents of structured analysis have attempted to dismiss alternative methods as “immersion” or “guessing”. Strict constructionists often define these other traditions so broadly as to encompass any technique other than a written matrix variant of ACH, due to the availability and emphasis of the literature attempting to copy Heuer’s excellent treatment of the subject.

Needless to say, this overly dogmatic approach is in our opinion unhelpful. We believe that while there are well founded conclusions which have emerged out of Heuer’s and his successors’ research, there are also hitherto yet un-quantified factors which contribute to the utility and validity of other approaches, and that further study is needed to identify and evaluate them in new contexts.

It is research such as cited above, as well as personal experiences in a range of very non-traditional and unconventional roles that strongly lends us to give credence to the possibility that there may “something different” with value to specific roles and functions within the intelligence community. To date, regrettably few academics in the intelligence studies field have shown any inclination towards such research, instead preferring the easier path of magisterial assertions that gloss over subtleties of craft in favor of strictures enforced with near religious intensity. And perhaps that alone is the most damning evaluation of most of those who claim to teach the art and science of intelligence after all.

We are hopeful, though, that as the intelligence studies field widens, and as new and more innovative minds enter the profession through non-traditional pathways – bringing with them their unique experiences and insights – that we will see new evolutionary progress in these areas.