Requirements management for the new generation

We frequently revisit the question of how the generation after next will change the activity of intelligence. It is ground that others have also slowly begun to come around to as the realities of the J-shaped workforce dawn across the community. As in many other areas of the workplace, generational changes tend to manifest themselves most visibly through technology. Thus it is the wiki and the blog which has more readily captured the imagination, while the less visible growth new models of ongoing production through informal networking brought about by the lists or the subtle changes in dissemination models driven by the Wire go largely un-remarked.

However, occasionally some changes are entirely striking not merely because of the realized technologies which have crept into the business of doing intelligence, but due to the aspirations of how a consumer would wish to see the activity done better (or at least, in a manner that better serves their particular cognitive and working style). These often frustrated desires for change are quite often poorly expressed, and rarely conveyed to the intelligence producers in any event.

Despite this, every once in a while there is something that a consumer may bring to the table from another life experience – typically outside of the intelligence domain entirely – that convincingly encapsulates both the shortcomings of the current approach and a possible model for examination of a new innovation. We enjoyed such a conversation the other day, the result of a highly technology savvy consumer’s attempt to grapple with a restricting “classic” intelligence structure that simply did not conform to the manner in which they had come to rely upon to satisfy other content needs.

We speak of course of the now standard approach to requirements management wherein a new consumer is asked to define collaboratively their Key Intelligence Topics (popularized in the private sector by the notable work of Jan Herring, having been drawn from the gentleman’s experiences as a National Intelligence Officer at a time when similar approaches drove national intelligence priorities). This is a valuable exercise, both for its role in drawing out a consumer’s concerns, pre-existing baseline knowledge, and assumptions as well as in helping set expectations regarding capabilities and deliverables. (There is a reason it first came into use and subsequently thrived after all).

The consumer in this particular case simply did not wish for the exercise to be a one time affair, or even something periodically revisited in accordance with some calendar schedule. Rather, having been taught by other information delivery environments that such a thing was possible, this decision-maker asked for a queue to which he could add, delete, and re-prioritize his own requirements as he chose. The model he cited for this was the Netflix video rental service, which offered this kind of flexibility.

This model is actually quite insightful. While one must overlook the key difference that good intelligence is not simply a pre-packaged mass market product sitting in some warehouse waiting to be sent out upon demand, and that intelligence questions should always be accompanied by scoping discussions, it is actually quite easy to modify and present such an interface for validated requirements that have been accepted by the intel staff (and to capture the additional conversations regarding objectives and terms of reference). The model also provided an unexpected benefit, in that the consumer already was willing to take into account the delay between request and production, having been trained to expect a few days turn around time for the postal mail delivery of the familiar red envelope. (For our British friends, the parallel to the equally classic red box presented to senior ministers is inescapable.)

Unfortunately such requests are infrequently captured by those in a position to implement the requisite changes even when a consumer does at last find a way to articulate what they are seeking. And while national requirements management exercises are far more involved things under the National Intelligence Priorities Framework, the back and forth of production on a smaller and more situated scale is entirely within the realm of the possible. Answering the inevitable next demand of delivering such an interface over their Blackberry – classified or otherwise – may however be entirely harder, at least in government service (commercial intelligence counterparts will likely be less constrained).

For those consumers that do not wish to actually use such an interface themselves, a periodic copy of the queue can be provided for discussion, and changes made by a briefer, unit manager, or other consumer outcomes oriented professional. In this it is little different than the classic requirements model, but it does offer additional opportunities for more dynamic engagement with the ongoing concerns of the policymaker. And we strongly believe that as the generational shifts also take hold within the ranks of key intelligence consumers, so too will the willingness to embrace already familiar processes in new contexts.

Understanding intelligence consumers

One of the persistent myths that we find too often in intelligence shops – particularly among editorial and production management staff – is the idea that an intelligence consumer cannot internalize complex concepts and ideas, and thus requires extreme levels of simplification. Where exactly this myth first originated is lost to time, but that it continues to dominate the thinking of those who ought to know better is perplexing to say the least.

There is a vast difference between simplicity of expression and complexity of thought, and all too often the search for the former renders incomprehensible the latter. One sees this especially in products which emphasize the removal of common terms of art – despite the utility such short phrases bring to both consumers and producers when referencing a body of other works. The use of such terminology does not excuse the author from placing them into a relevant context, and using such context carefully within the structure of the products argument. However, all too often editors and others involved in coordination reject such expressions of brevity with blanket dismissal, citing perceived military origins of a phrase in many cases (as much of intelligence’s activities derive from war fighting origins, no matter what the current application), or as “excessively” technical. This is a grave disserve to the reader, who if they are not acquainted with a key concept associated with an intelligence issue, should be introduced to that background rather than left to assume understanding of a distorted turn of the phrase.

More and more, the results of this tired old trope become a primary reason why many consumers cite finished intelligence as irrelevant to their actual responsibilities – all the moreso when those consumers hail from segments of the community’s readership that may typically not have previously relied upon finished intelligence as part of their decision-making processes. Given the increasing levels of education demanded of even middle management, let alone the kind of experiences a typical senior might lay claim to, it is understandable that they would reject any attempt at intellectually coddling at the same time they are expected to be informed by a product. After all, it is not uncommon to be briefing or writing for a consumer with post-graduate education and longstanding experience grappling with difficult legal, business, and public policy issues. And given the number of lawyers that now make up the ranks of senior government and industry leaders – it is difficult to insist that an individual who might in another context be expected to rapidly skim and comprehend the most challenging of legal briefs must somehow never be troubled with a product written at a level higher than the typical newspaper’s 8th grade education standard.

This also contributes, in our view, to the reasons why consumers typically emphasize the value of many forms of “raw” and other internal intelligence reporting. These latter products tend to be authored for the primary readership of other intelligence professionals, and is therefore not “simplified” in the fashion one comes to expect from the editing and production process. It is no wonder then that many consumers – in some cases now more experienced than the analysts and other staff which serves them – prefer the “roughs”.

In the end, the manner in which a finished product is best crafted demands that the author understand the reader – and not simply rely on old maxims of some journalism editor’s style guide, applied without real consideration.

Counting the minutes

Those with even a passing interest in counterproliferation issues will no doubt be well aware of the Bulletin of the Atomic Scientist’s infamous Doomsday Clock. Tao Security examines the implications of using a clock to convey threat information, and in particular the implications of shifting what was originally designed as a publication art into a watch and warning mechanism.

Many of these lessons, we think, could be taken to heart by those seeking to implement an ever wider array of threat level displays for almost every new issue that arises in a multi-agency or cross-account context. Too many of these efforts seek to differentiate themselves by creating their own standards, hoping to be cited with the same frequency as the more recognized measures of warning.

However, all face the same questions of setting an initial baseline, and of calibrating the changing conditions to the consensus of expert judgment based on imperfect and often contradictory indicators. These by themselves are not intractable problems – in fact, they are quite familiar to those routinely engaged in the business of analysis. (Although many now pushing for new threat level measurements do not themselves come from the analytic tradition.)

What complicates these new efforts is the effective communication of these measurements – often done in color coded fashion (to mimic the DHS National Threat Advisory), or through some combination of alpha-numeric or word designators. These are displayed variously as bars, speedometers, and a variety of other graphical tricks pulled from simple spreadsheet charts so favoured by accountants. And we can think of few less productive ways to spend the valuable time of intelligence professionals than group debates over the exact shade of color, or font, of one of these new graphics.

Effective warning in intelligence is a difficult task that is as individual as the consumers that it serves. We think the clock model, and its counterparts of the speedometer and bar graph, deserves to be left to rest. In much the same way as the intelligence community has moved away from numeric designators for source reliability and accuracy and towards more descriptive paragraph summary formats, it is long past time to reconsider how warning conditions are conveyed to decision-makers in a more robust fashion which clearly brings across not only the underlying indicators but also the key changes which drive current judgments.

Good old fashioned military analysis

One of the persistent problems in most mainstream media coverage of ongoing matters of national security interest is the continual failure to understand even the most basic factors military analysis. Evaluating weapons system effectiveness, for example, requires parsing through the marketing hype of the manufacturing country (or company), and examining the hard details of design, employment, and logistics that are essential to understanding how that system will perform in the field.

The recent abysmal state of press coverage is demonstrated amply by case studies of Syrian and Iranian air defense network, the “new” re-manufactured Iranian tactical fighter aircraft, Russian fuel air explosive development, and any of the missile test dog and pony shows conducted by the Iranians, North Koreans, or non-state actors such as Hezbollah and HAMAS. Indeed, this was perhaps the dominant media game of the 90’s, in which pundits came to prominence speculating about campaigns which would never come to understand during Kosovo, East Timor, and even the Desert Fox air strikes against Iraq.

This is not a new phenomenon, and certainly not limited to the areas of military interest given the wide range of other areas in which modern journalism fails utterly to provide an accurate comprehension of difficult subject matter – such as medicine, economics, or law. It is in part the raison d’etre for a whole range of custom open source intelligence services – from the Economist Intelligence Unit to the venerable Jane’s Information Group. Indeed, it was in the Kosovo conflict in which the Stratfor notably (or infamously) first made its bones.

The blogsphere is changing these dynamics, however. Now, within a very short amount of time, substantive analysis is published by a number of individuals possessing real expertise and experience on the matters at hand. Whether it is to take down a fabulist or fauxtographer, or to dissect the propaganda claims of a potentially hostile regime – the distributed and self-organizing interests of milbloggers (and experts in a range of other fields) is producing a kind of open source analysis capability that one could only have dreamed about in earlier years.

And when one considers the old OSINT maxim that an author will only ever write less than 20% of what they actually know about a subject, one begins to glimpse the potential of this population - given a means to organize and direct their “hobby” level efforts in a larger architecture of participation.

At the same time as this is occurring in the open environment, the intelligence community has seen a dramatic decline in its bench and depth, of those products which are produced as “shelfware”. We use the term not in a pejorative sense, but in recognition of the very real need for a baseline reference set by which current issues and future estimates made be assessed. Perhaps the better term we may coin for this class of intelligence are “enabling products”. Many commentators within and outside of the IC have bemoaned the shift of emphasis and resources away from strategic and long term production to the ever-expanding needs of the tactical and immediate. And while we support a community which can and will run to the sound of the guns in order to support the warfighter’s ongoing operations, the length and demands of the Long War will have to force the community to examine how it can serve both ends effectively. To be sure, enabling products has true limitations: whole ranges of issues and concerns simply did not have the same priority – or even the same conceptualization – before developments in the field made them central concerns. After all, in 2001 it would have been quite hard to predict that the humble IED would be the primary weapon of strategic influence, or that the Katyusha would become the strategic equivalent to a stand-off bombardment capability of a type seen previously only in the War of the Cities.

It is in this environment that we look around with the growing realization that there may be an unprecedented opportunity to change the traditional model of analysis through new partnerships with those outside of the core community. There are a host of concerns – from dealing with statements intended to influence rather than inform, and some very valid counterintelligence / denial and deception issues, particularly if requirements and questions become too widely known or too narrowly specific. The logistics and coordination of such an engagement – even a in narrowly bounded iteration – would be far from trivial. Yet the potential for leveraging the contributions of the kind of capability that occasionally emerges in ad-hoc fashion at present into a more dedicated and systemic resource is frankly one that would be worth at least an exploratory effort by certain parties with natural interests in the area.

We look forward to a day when good old fashioned military analysis is easier to come by and of better quality, based on such efforts, than has regrettably been the case in recent years.

Fighting plagiarism beyond academia

We have lost track of the number of times in which we have had to gently (or perhaps more sharply, in a few cases) remind colleagues that the admonition against plagiarism is not merely a conceit of the academic world. Unattributed verbatim copying of others’ text is not an acceptable practice given the importance of national intelligence, and the need to be constantly vigilant against the possibility of deception.

If nothing else, we should think that the rather high profile plagiarism case of the British intelligence “dossier” in the lead up to the 2003 start of operations in Iraq would have provided ample warning to those in the community of the dangers, and the embarrassment, of recycled texts. Regrettably, this is not always so – and with the advent of so many new intelligence shops (particularly those at the state, local, and industry level within Homeland Security), this is one of the lessons of good tradecraft all too often forgotten, or never learnt in the first place as struggling new analysts attempt to meet implacable production demands by shuffling and re-shuffling derivate reporting into ever more diluted distributions.

We recall one particular case in which a senior executive would routinely hack together documents consisting of nothing but large chunks of text copied and pasted into some semblance of a paper – often with inexplicable digressions and other non-sequiturs as the inevitable result. (This individual’s behavior was driven in no small part by the common practice in OSINT of the 90’s – where he spent a good deal of time – of assembling “summaries” consisting of the full text of various news articles pasted into a single document – a glorified clipping service. Bad habits die hard, even years later.)

We note a recent case of explicit sanction against such behavior in the legal profession, courtesy of the Volokh Conspiracy. It seems one lawyer, attempting to shortcut his workload (and no doubt generate additional billable hours), was caught recasting the text of legal articles as “arguments” in a brief presented to the courts. We were not aware that this was considered not only gross professional misconduct but actually the offense of misrepresentation – as well it should, upon reflection. In this case, the offender was sent to remedial law classes – no doubt as much for personal embarrassment as professional education.

We believe it is time that managers in many intelligence shops institute similar sanctions. And of course, intelligence studies academia must instill into its students the absolute taboo against plagiarism – including even too frequent copying of attributed data such as the citations lists noted in the Volokh cases- something that has not always been a priority in many classrooms. We recall recent products which included verbatim sections of popular community reference resources such as the Factbook… and while one can argue whether such material is plagiarism under the legal precedents noted above (even if clearly marked as to origin), it is certainly a poor production model.

Consumers do not need recycling of the dry statistics of other shelfware products – they need unique insights, clear writing, and strong assessment based on solid sources and analytic tradecraft methods. Padding merely detracts from these objectives, and is certainly achieves the opposite of the presumably desired effect of displaying a semblance of rigour not otherwise earned. Students do not need to be taught to fill in checklist style product templates in which form trumps content, but rather they should be encouraged to choose among the many models within the community and develop their own unique approaches consistent with the blending of best practices from multiple agencies and traditions.

Brandy with the Mechanical Turk

We have long been staunch advocates of commercial overhead imagery systems, and from the earliest days of their uncertain launch and performance we would vocally proclaim their potential to any and all – most of whom who would darkly mutter about NIIRS, classification, and mission availability.

However, commercial IMINT has so far exceeded any hopes we could have dreamed in the early days of the 90’s, having been brought to widespread use by Keyhole / Google on a scale previously unimaginable. The resulting sophistication of new processing techniques – from humble mash-ups to the surprising benefits of the open source model of many eyes – has given us great reason to reconsider the infamous “bottlenecks” in the architecture of other national technical means. Perhaps it is not the nature of the problem that has so bedeviled those of us seeking to expand the percentages of raw take examined for interpretation and analysis, but rather the model of the solution which is defined so very much by implicit and unquestioned assumptions.

The strange case of a missing billionaire “adventurer” has given us further evidence that other production models may well exist. High resolution imagery was acquired by a private donor’s tasking of the DigitalGlobe constellation, and it has been populated into Amazon’s innovative Mechanical Turk system – a fascinating technology designed to facilitate the development of architectures of participation to effective harness ad-hoc volunteer efforts. The system’s name, of course, derives from the famous chess playing automaton of the 18th century Austria-Hungarian court.

We wish the searchers bonne chance, for we know well the damnable difficulties of wide area search problems. We would also hope that any of our readers in the professional geospatial community might consider lending their unique talents and expertise to the search. (Those wishing to do so may contribute at the Steve Fossett Missing Mechanical Turk page.) We hope that a suitable flask of good brandy – in the model of the Saint Bernard rescue dogs – is close at hand on the day of the searcher’s success.

The effort is a dramatic example of innovation, however, that will have utility to the GEOINT community long beyond this crisis. This may well be the esteemed Mr. Fossett’s greatest legacy to the intelligence community – although we do hope he returns safely to provide many more candidates for nomination. We could easily see a trial program initiated for similar analysis of open source imagery and geospatial extraction, through an albeit less public architecture. USGIF certified schools and other intelligence studies programs would be a natural first home to such an experimental platform. This may also bring IMINT and geospatial analysis instruction into the mainstream of the intelligence studies curriculi, despite having long languished in the backwater of a narrowly perceived technical corner.


h/t Slashdot

He who owns the platform, redux

The following item cited by SmartMobs reminds us how very applicable the earlier noted lessons of control achieved through better processing, exploitation, and analysis – rather than mere ownership or creation of the raw data – are when examined across disciplines and INTs. In this case, they examine OSINT and social network analysis applications based on user-generated content.

While in the civilian world, that buzz phrase may drive much investment; the quieter versions of participatory architectures found within the community are also facing similar issues in the strange and uniquely competitive marketplace between agencies, programs, and dueling subject matter experts.

But what may seem a short term “victory” in the turf wars for a specific program may in fact be the roots of its downfall. Even the “best" practised, uber-authoritarian walled garden of “collaborative expertise” becomes essentially irrelevant if it strangles the contributions of its experts through attempts at control based on philosophies of ego or proponency – something we have seen in many of the non-inclusive small wikis and other similar “sharing” platforms being generated both inside government and by its academic partners, attempting to copy the concepts of Intellipedia and other like tools. In the worst case scenarios, over-controlled efforts die as their contributors invest time and effort elsewhere; or are fatally crippled as new contributions originate only from those without experience or access to more productive and vibrant environments.

Even experiments which fail teach valuable lessons. However, the manner in which these lessons are taught, and the next generation learns, will dictate whether the efforts are valid. Certainly though, the energy and drive of the individual analysts on the line will count far more in the long run towards developing and validating new techniques for distributed collaborative production – and capturing the ephemera of those activities as lessons learned and best practices should certainly count far higher in the schoolhouse and literature’s priorities than simply creating yet another “me too” attempt at a technological fix.

Mobile consumers

The increasing prevalence of real-time handheld mobile email solutions of the crackberry variety is certainly a well entrenched fact of life in the Beltway and other highly urban areas. The following item highlights interesting factors that impact decision-makers who receive information via such channels, versus more conventional communications methods (the political aspects of the specific situation aside, and frankly entirely irrelevant to our discussion).

Within selected organizations, the use of the crackberry to handle almost all material (at lower handling levels) is legendary. Just as community best practices changed to reflect the differing production requirements embodied in web-based dissemination, so to one would think changes are due in the face of these new changes. However, few shops have given any serious consideration to these issues, in no small part due to the fact that Blackberries and their ilk are “just another email option”.

Ironically, it is the homeland security organizations – particularly those at the state and local level – which are leading the way in changing both products and consumer expectations to embrace the mobile intelligence consumer. They face similar, if not sometimes more severe issues of security, handling, and control; but the lack of structure and technological support which has so plagued them in other areas for once appears to be working in their favour.

It remains to be seen what lessons might be learned from such dynamics elsewhere in the community. There is a serious gap of real and systematic research into the effects of new technologies on consumers, and the solutions intelligence producers are finding to help meet their clients’ needs in this new environment.