On the origins of competitive intelligence

We rarely touch directly upon our commercial brethren in the field, for we feel that much of our writings have relevance for this audience in their original form. We do not merely seek to write for a single group within the profession, and are fortunate to count among our readers many from the financial, pharmaceutical, and consulting set. We see their challenges and travails in applying the art and science of intelligence in often untrod territories as not so very different from those faced by a fusion center’s staff, or the senior intelligence officer who must stand up a new account-specific cell.

We have been too often struck however by the insistence of some in the commercial side of the profession that their work is entirely a new creature, divorced utterly from the precedents of the community that came before it. While we are the first to grant them respect for the unique conditions under which they operate, and for the sometimes serious challenges those conditions create, we would beg to differ most strenuously that the history of the community does not matter to them. We can point to the lessons of intelligence / decision-maker relationships or the experiences of communicating inference, evidence, judgment and uncertainty, along with any of a number of other areas in which the history should prove instructive in first principles.

There is nonetheless a too stubborn belief held by many competitive intelligence types that they have to be a “new” field, in order to carve out their place in the world and their relevance in the modern enterprise. And it is this insistence that makes us laugh, and reflect upon the length of the history underlying commercial intelligence activities. After all, most current literature cited by practitioners in the business intelligence world are indeed not that old in comparison to the national security side of the house, although we would question anyone citing what is now nearly 30 year old material as “new”.

Older yet are a few texts which for too long languished undiscovered in the stacks, and rarely cited in the literature – such as the 1966 volume Business Intelligence and Espionage by Richard Green, as explicitly a competitive intelligence work as any we have seen. The book also cites information from a series of business intelligence conferences, held starting in the 1930’s. These together create a timeline nearly equal to that of the better known literature of the foreign intelligence disciplines.

We have also written before of the excellent work being done by a number of the leading information industry firms of the day, in digitizing and disseminating the entire contents of a number of libraries worldwide. Among these millions of texts we have discovered quite a few forgotten gems of late, including works which point to the far older origins of competitive and business intelligence. The oldest volume we have yet uncovered is a series of reports first published in 1913 from commercial intelligence officers who advised the British government. The piece was presented to Parliament and was made available for general sale through the Stationary Office - though the level of interest was not recorded as far as we have yet seen. The text is in the form of correspondence, and is primarily descriptive in nature, providing quantitative information as well as some attempts at analysis. All in all, quite dry, and but for its historic value would not make for very interesting reading.

But it is precisely the historic that offers a glimpse into what we see as an earlier, parallel evolution of the profession. The early commercial and economic intelligence work of governments translated near seamlessly into the business intelligence of industry in its earliest incarnation. However, over time this side of the profession experienced its own wave of privatization. We have no record of the countless business intelligence shops that must have existed from the 1930’s onward – the turmoil of the depression and WWII no doubt radically reducing the number of active shops for a time. But simply because history has been lost does not make its successors “new” or unprecedented. It is thus the old lessons we would seek to have acknowledged – even if they cannot be entirely rediscovered. No doubt however there remain individuals alive who could contribute to the oral history, if nothing else – although we also strongly suspect somewhere in nearly forgotten corporate archives may be buried a few more gems of the early shops; should some young intelligence studies student seek to make his or her thesis through pursuing both the written and verbal remnants of this vanished era.

For those debating the current trend towards the privatization of intelligence in what are considered “inherently governmental” functions, the history is also instructive. At one point, it was deemed entirely a government responsibility to provide commercial and economic intelligence to the leading firms of the nation. Of course, in a globalized world of multinational corporations increasingly removed from industrial age models, that kind of intelligence support is not only infeasible but a questionable public policy. The private sector stepped into the breach to meet these demands through a variety of mechanisms, not all bearing the label of intelligence. In some cases, successful commercial shops have seen the cycle come full circle, as government agencies now purchase their finished intelligence products as the best available source of information and analysis.

We wonder how many of the intelligence functions now deemed the responsibility of government might also shift towards an entirely privatized existence over the course of the 21st century. And for those that protest such a course as unthinkable, we would gently remind them to look to the flowery language and proud signatures of those commercial intelligence officers of a century ago. They too no doubt thought their world forever the domain of government.

Further to the scholarship of blogs

We recently noted the ongoing discussion regarding the impact of blogging on scholarship in a number of fields, and to be honest, it is a matter we had rarely had occasion to reflect upon. In our own experience, the continuing conversation of the blogsphere (in both public and other networks) is to a great degree the enabler of new ideas and further discussions across a wide range of environments – from the formal journals to the internal ephemera of countless working shops.

Thus it is with some puzzlement we have observed the response to the discussion of the intellectual merits of scholarly blogs by those who would claim to be enforcing some alleged degree of academic standard. We have quite a bit of recent correspondence on this issue, such that we believe it merits wider response in a more general form, if only perhaps to advance the conversation more widely among those with new perspectives.

In some ways, we see the intelligence studies blogsphere - nascent though it currently may be in comparison to other more established fields such as law) - as the modern day equivalent of the circulated letters that used to pass from hand to hand amongst the academy of old – only conducted on a far more rapid timeline to a far greater degree of visibility. It was not uncommon for such correspondence – particularly involving multiple thinkers over time – to be used as the basis for a range of other publications. We still frequently encounter writings from this era, in which the correspondence or private discussions are footnoted accordingly.

We see no great difficulty therefore in extending the same consideration to the correspondence of the blog. Indeed, given the volume of unique and primary source information that is now conveyed by the authors of many blogs – including many new forms of online journalism, research, and expert discussion – it is incredible to us that such material would be ignored in more traditional modes of the literature. To do so would be to dismiss important aspects of contributions by those on the ground in conflict across the globe, or otherwise engaged in the practical pursuits of the profession.

The sharpness of the recent discussion reminds us of the vitriol also displayed towards other forms of evolving academic practice, such as the online engagement of distance learning studies; or the disdain for wiki-based collaboration models. While the discussion of the academic uses of the ever controversial Wikipedia (and its counterparts of technology) is in our minds an entirely different matter – being a thing of more ephemeral nature than the recorded conversations of the blog – we see many of the same rice bowls threatened by these technologies. We also find ourselves distressed by the increasingly fine distinctions of supposed authority that too many in the intelligence studies field now are attempting to impose, in order to preserve those rice bowls – such as one recent assertion that the field’s longest standing publications such as Studies or the JMIC / (new) NDIC papers are not “real” academic works, and therefore could not “count” in the eternal publish or perish world of those seeking tenure.

These arguments are very much the opposite of academic, in our view. The encouragement of critical review and analysis of any source material is fundamental to the practice of intelligence scholarship – and even moreso to the practice of the profession such scholarship purports to study. Credibility is not established by institutions alone – a point reinforced recently by the scandals which have afflicted many otherwise respected establishments - but rather more importantly through the continuous evaluation of ideas and their expression.

This debate will likely seem very odd in only a few short years hence, as the pace of evolution overtakes the academy far more rapidly than those within its walls are prepared yet to grasp. And there will be yet those who evolve and master the currents and complexities of this new domain – where the words of a forward deployed officer carry more impact to the literature than a number of dry treatises recycled on dusty pages, or where a student’s work may offer more promise than that of a tenured professor. Those members of the academy which can adapt – as many within law schools clearly have – will find themselves in a far different future than they might have otherwise envisioned. It will still however be a future in which the intelligence studies academia has much to offer, if it is willing to join the conversation as it is actually happening in the larger world, rather than as they might wish it to remain in their hermetically cloisters.

Examining intelligence activity in counterinsurgency units

Via the incomparable Small Wars Journal, we note a currently serving officer’s contribution to the literature in the form of articulated longing for the solutions that can help end his ongoing frustrations with the intelligence system as it is now structured. Among Captain Tim Hsia’s valid and well articulated points include a call for better portal management, a discussion of potential applications for market data mechanisms, and some interesting critique of the narrative format of Intellipedia and A-Space from the perspective of direct support to ongoing counterinsurgency operations.

We empathize with his frustrations, although we do not know that another “central” portal system or new repository is necessarily the answer to the ever expanding complexity of classified holdings and networks. The captain’s pain, we think, would better be served by a situated software application – a portal or other tool that could create, for his unit and those like his serving throughout the far reaches of the Long War, the functional appearance of centralization, at least insofar as this means the kind of “one stop shopping” for products in support of ongoing operations that seems to be needed. Frankly, we believe that the creation and maintenance of such a portal is a clear role for a reachback function – if such an element can identify and stay current with the constantly shifting needs of those on the ground.

Part of the pain also comes from cultural barriers to information sharing – and not the classic interagency turf battles that we are used to discussing in intelligence studies. Intelligence support in operational environments is typically least focused on formal production and publishing, spending its energies on communicating with its primary supported consumers (as well it should). This however creates a vast sea of products which are rarely formally disseminated beyond a specific unit, or perhaps its replacements. Certainly, a number of this would be no means be up to the classic community publication standard – but in many cases, they were not created to endure for more than the next 12 hour battle update cycle. Much of this support is also a highly verbal process, in which judgments and analysis is conveyed through discussion and debate, rather than in formalized bullet point.

Under the kind of time constraints a typical forward deployed intelligence function must operate, this lack of emphasis on publication is entirely understandable. What is most remarkable, perhaps, is rather the number of shops which do manage to maintain distribution of a high volume of products to other elements in the community, in addition to their pressing and immediate needs locally. Given the importance of intelligence obtained through non-traditional (and non-community) assets and methods within counterinsurgency operations, one cannot emphasize sufficiently the unique value the investment of that time and effort can bring to the community, in ways that are difficult to predict in advance.

One of the ways in which the reachback elements within the IC could better encourage and support the distribution of such products is to provide a better platform for lightweight publishing and databasing – one that will support the forward shop’s immediate needs, but also offer options for wider sharing of extant production, in whatever form it might take. The demand for a geospatial component – perhaps even if only through meta-tagging and geospatial search and display – is also clear. While a number of existing portals might be adapted to support this functionality, the typical sharepoint style (regardless of underlying software implementation) definitely lacks the situated elements that forward units most seem to rely upon. Likewise, the suggestion of the market tracking model is interesting as a potential model of organization – and outside of predictive trading style applications, the first time we have seen it floated. It is well worth exploring - paging Bloomberg?

We are grateful for the time taken by Captain Hsia to help advance the literature of intelligence, and particularly to help focus attention and debate on the counterinsurgency account. We also commend his choice of venue for publication – the Small Wars Journal is certainly emerging as one of the pre-eminent venues for these discussions. We definitely look forward to further contributions from both the author and publication in the future.

Traffic analysis

Hauntingly beautiful in its own right – the following is an excellent example of the power of visualization to make sense of complex patterns within extremely large datasets; in this case, aviation flight records.

There are equally haunting displays of the exchange of information, of light, and of motion, in other areas which are so close the art and science itself. In the words of the master (still accurate, even three decades on):

“Program a map to display frequency of data exchange, every thousand megabytes a single pixel on a very large screen. Manhattan and Atlanta burn solid white. Then they start to pulse, the rate of traffic threatening to overload your simulation. Your map is about to go nova. Cool it down. Up your scale. Each pixel a million megabytes. At a hundred million megabytes per second, you begin to make out certain blocks in midtown Manhattan, outlines of hundred-year-old industrial parks ringing the old core of Atlanta. . .”

Sometimes we forget the unintentional glimpses into the very nature of the human activity that our unique profession affords us. In deep time, these may be the closest thing to the historical record of a whole range of otherwise unremarked aspects of our day.

UPDATE

It appears Zenpundit's thoughts have also drifted towards the visual, with the discussion Cognition of a Society of Visual Imagery, building upon a post at Glittering Eye. Perhaps his blog redesign (utilizing, as we understand it, the excellent talents of his wife's firm) has contributed to his greater appreciation of the aesthetic. Certainly, we his readers have benefited.

Case studies with enduring legacy

There is much to be thankful this holiday season, and it is natural at such a time to reflect back on the conflicts past, and those lost in the course of these conflicts. It is also a time to reflect upon history within the intelligence community, as our first – and for nearly 60 years, our most severe – intelligence failure at Pearl Harbor falls within these months.

Recently, new allegations have surfaced regarding what some consider to be another greater intelligence failure – the Allied ignorance of the machinations of death that were the Holocaust. This has long been an area of immense historical controversy, and present a number of opportunity to examine intelligence / policy relationships, the effects of cognitive bias, the questions of imagination (in thinking about the unthinkable), and (due to the importance of ULTRA decrypts in the overall reporting stream) decision-making in the SIGINT environment.

In short, the new claims put forth in the Hebrew language book Pazner: The Man Who Knew, are centered around supposed HUMINT reporting containing explicit discussion of the construction of camps intended for extermination operations in German occupied Eastern European areas. The reporting, alleged to originate in July 1942 from an unidentified German officer source with what was claimed to be good access, was passed through a Swiss intermediary to a Jewish Agency official. This reporting was subsequently provided to Allied intelligence, and may have even made its way to the highest echelons of the British government.

If true, it is an interesting addition to the historical case study of the Holocaust warning problem. One raw report does not warning make – particularly third-hand HUMINT from unevaluated sub-sources. However, a HUMINT source would have been encumbered by only a fraction of the difficulties of dissemination that the information provided by the ULTRA channel would have imposed. It is possible that such reporting could indeed have been the best “public case” material that could be used to support strategic communications and information operations pressure against the Nazi regime. Indeed, the source is alleged to have even proposed such a use of the information himself, recommending daily BBC broadcasts warning against the commission of war crimes. A later very similar reporting stream, which surfaced through the Riegner telegram, did become the basis of the US acknowledgement of the genocide in November 1942. Public diplomacy at the time had impact on the Nazi pogrom - and given modern experience in Bosnia, Darfur, and elsewhere - it is indeed questionable how effective even widespread public knowledge would have been in halting evil intent.

The handling of this information joins the already extant list of intelligence controversies from other collection disciplines – from the ULTRA COMINT to strategic air IMINT. The ultimate questions that are invoked, however, are ones of policy and operations – not merely intelligence. And these all continue to be well worth study – particularly in the face of the continued threat of genocide throughout the world.

Differential problem solving

Business and competitive intelligence practitioners are one of those often overlooked sources of insight into analytical tradecraft in situations, and with individuals, very different from those typically encountered on the normal pol sci / national security track. Many of those insights have surprising cross domain relevance to the intelligence community proper.

Thus it is with the following item examining the behaviors of various (non-intelligence) technical minded types when confronted with an analytical problem. The differences in mind-sets, approaches, and even processes employed by the two groups make for a fascinating little vignette regarding comparative styles of critical thinking. (While we may object to the conclusions drawn in many other items on this particular blog, we do occasionally find an item or two of interest. And of course, given that the author resides in Kent in the UK, we could not help but notice the blog.)

We recall a similar little thought experiment that was routinely conducted during a strategic intelligence analytic tradecraft class once commonly offered around the community. The exercise involved a group of practitioners and managers, usually from very different organizations and entities, trying to tackle a new “emerging” target account from soup to nuts – requirements definition, collection management, and analysis and production planning. We likewise recall the very differing approaches and styles that surfaced during those discussions. At the time, common wisdom tended to chalk this up to institutional culture – and in particularly, the reinforcing power of organizational stereotypes (and expectations of those stereotypes). A more subtle interpretation would be to examine the extent to which individuals came to those organizations, and to succeed within them, that were more likely to embrace those cultural norms and even stereotypical responses.

The benefits of this kind of cross domain discussion (if you can keep the participants from killing each other) is perhaps one of the strongest arguments we have heard voiced regarding the need to break the old ivory tower model, and to integrate collectors, analysts, and operators throughout key accounts – in close proximity to the policymakers.

The changing of the intelligence studies (public) blogsphere

It is against the backdrop of our continuing discussions of intelligence studies and the impact of blogging that we note the passing of two blogs which we have had occasion to enjoy over the past while. The World is Grey and Fast Squirrel have recently chosen to close up shop. We respect their decisions, and understand entirely – as much as we will miss their contributions to the public literature. They join the now defunct JMJ Blog, Intelligence Analysis & OSINT, and Robot Economist among those recent intel, and intel related blogs which have gone dark.

Blogging – especially the personal, hobbyist kind – has always been a dynamic enterprise, with a constantly shifting menagerie of new faces. Even among our own humble skunkworks here at KI, we have seen contributors come and go. The public side of intelligence studies blogging is perhaps more prone to this than other aspects of the distributed conversation, if only due to certain unique features of the constrained information environment. After all, some may find it difficult to accept that the interests which drive the majority of their day and professional energies must be totally ignored in public discourse, and that the focus of online writing must remain academic, historical, and high level. Doing so in a manner that maintains both relevance to the reader, and the satisfies the intellectual interests of the authors, can certainly be a challenge for some bloggers. (We are fortunate here at KI that our interests span such a diverse range that we find no difficulty exploring areas of the field in which there can be no conflict of interest.)

At the same time as our old (virtual) companions have closed up shop, a few other blogs have come to our attention that appear to be fellow travelers in the revolution in intelligence affairs. These include the Intel Fusion blog, and the AFCEA’s MAZZ-INT commentaries. It is not surprising that several others hail from overseas (allies), or the private sector’s competitive intelligence / business intelligence spaces, such as the Singaporean National Security Intelligence, the Italian Intelligence & Security Analysis, the Competitive Intelligence Marketplace, or Jens Theime’s weblog. This does not make these potential contributions any less valid, of course – but such material does perhaps fall into the nebulous category of comparative intelligence studies.

Given the difficulties of public blogging for many intelligence professionals, we do expect to see the expansion of the activity on other networks rather than public pieces by a number of those authors seeking to enter the conversation. But that is a discussion for another time and place entirely, we should think. Our only hope is that many of those choosing to blog in more rarefied atmospheres recall the original admonition in Sherman Kent’s call for the literature, in which he urges those with an inclination to write not merely on their target or issue, but on the nature of their shared profession and its tradecraft. It is all too tempting in an environment where the details of one’s accounts may be shared more fully to explore those details and their implications, rather than the higher level abstractions of process, patterns, and transformation. However, it is the latter concerns that will truly respond to the Imperative laid upon us all, and help to chart the unknowns of the profession’s continuing evolution.

Blogs and (intelligence) scholarship

Via the TaxProf Blog, we note the proceedings of a conference designed to examine the impact of blogging on the study of law. The parallels of many of the key arguments for the intelligence academy are quite striking indeed.

Recent years have seen not only the dramatic expansion of the intelligence studies field, but also attempts by a number of parties to impose their own control on the manner in which the field is evolving. These efforts have varied from proposed standards bodies to commissar style denunciation of new programs as mere “training” for vocational skills. Some are motivated by the best intentions of furthering professionalization, and some are regrettably little more than rent-seeking.

We have been greatly surprised that the expansion of the actual work of teaching intelligence has not led to nearly the flowering of the literature of intelligence that one might otherwise expect. For those most part, intelligence faculty simply do not publish – in the community’s own organs, or in the public journals that have arisen in the space. Now this may be in part explained by the moribund state of the field’s traditional publishing opportunities – but we doubt such an explanation complete, especially in this age of lightweight digital dissemination such as SSRN. Nor are we content to ascribe the lack of new material simply to the effects of tenure – though many a sleeping backwater might offer tempting distractions to those who enjoy lifetime employment, too few intelligence studies educators actually have their tenure in hand (not least due to the relative newness of many intelligence studies programs.) Likewise we find it difficult to assign blame to the consulting pathways that many intelligence professors also pursue, given the importance of networking and thought leadership in building intellectual capital for consulting assignments. (Indeed, we find that many of the scholar/bloggers of our acquaintance cite their blog as a more productive source of engagement opportunities than the institutions which supposedly provide them a place to stand respectably in the public square.)

There are many arguments both for and against blogging in pursuit of furthering the intelligence literature. In our opinion, not everyone is cut out to blog – and many, for reasons of classification and operational security, cannot (at least in public form) for entirely valid and understandable reasons. Yet this is no excuse for far too many intelligence educators – from those who work primarily if not exclusively in the unclassified, open source realm to those many now that focus on the pre-induction student pipeline at the university level.

We would strongly second the warning first given to the legal scholars as also quite fitting for those in the intelligence studies field: evolve or die.

The form into which intelligence studies evolves does not have to be a blog, but it has to be a mechanism which will advance the literature and the scholarship - and in a manner which has relevance to the native 21st century intelligence community, not merely the quaintly archaic exchange of dead trees that marked the industrial age era. Evolution does not come without risk – of professional differences and disputes, of public errors and their required corrections, or of the revision of one’s thoughts and writings in light of debate and further discourse. But in this it is no different than other forms of scholarship – merely conducted on a shorter timeline, with a self-selecting audience that is in many ways far broader in expertise, and often far more deeply studied, than the author might expect. Yet this is how the profession grows, and it is the imperative that was laid upon those who would make claim to its examination to pursue that growth.

Living intelligence history through hobbyist cryptanalysis

Following up with the latest in the recent stream of news regarding the ever so eccentric world of code breaking, we are both impressed and amused with the effort at Bletchly Park to recreate the machines that automated much of the attacks against German cryptosystems – and arguably shortened WWII by an incalculable duration. The reconstructed Colossus will test its mettle once again versus the Lorenz cipher, code named FISH.

The original FISH break came about in no small part due to operator error - which involved retransmission of the same lengthy message twice, using only slightly modified cipher settings. Let us hope that similar fortune favours the boffins of Station X once again.

Given that the recreated test also involves a comparison against a modern virtual emulation of the Colossus system, we cannot help but wonder what the results might be if a modern botnet-based cryptanalytic attack was to be added to the race? To be fair, we suppose a virtual analogue of the Y Service intercept system would also have to be crafted – perhaps paired with a software defined radio link or a TCP/IP intercept capability of some flavour. Of course, we would rather prefer the pretty young female clerks that used to operate the radio sets...


h/t Futurismic


UPDATE:

The rebuilt Colossus worked as expected, and achieved a successful break of the encrypted transmissions in a little over three hours. A modern computer analogue did nonetheless manage a successful break faster. Joachim Scheuth, a German cryptologic hobbyist, is to collect the first round of drinks at Bletchley.

However, the exercise did demonstrate the sheer difficulty of SIGINT – particularly in the interception of the weak radio signals of the day using contemporaneous equipment. Something to ponder in this age of far more sensitive systems, and far more elevated expecations.

Lessons of the strike

We have never been much a fan of unions in this modern era of incredible individual opportunity and rapid innovation – qualities seemingly quite at odds with the behavior of most forms of organized labor. We do however freely admit that unions have their place in a competitive market economy so long as individuals are free to choose to organize (or not) as their preference dictates, free from pressure or intimidation. For a variety of reasons, however, we have never had much call to consider organized labor within the context of the intelligence community (though the actions overseas unions were indeed a prominent feature of political analysis during the Cold War.)

Thus, the ongoing Hollywood writer’s strike is quite unfamiliar territory to contemplate. We have long known that the studios which drive that town are essentially specialized forms of financial vehicles designed to leverage investment against the high initial costs of media production and distribution against a potential return measured in popularity (and attending profits). In the course of our musings on the death of other forms of traditional media – particularly those of the fishwrapper variety - we have enjoyed the analysis of various contemporary commentators and pundits who sought to describe the impact of the digital revolution on these entities’ future. However, in the past they had seemed sufficiently disconnected from the concerns of the community’s own revolution in intelligence affairs that we could discern no potential lessons learned – with the possible exception of the fiasco that digital rights management (DRM) implementations have become, and the attending incentives for new cryptanalysis and covert communications technologies that emerged for defeating such DRM and related copyright enforcement systems.

However, the serial entrepreneur (and philanthropist) Marc Andreessen offers up a new piece regarding the strike that has spurred us to reconsider that opinion. His take on the situation is a classic example of opportunity analysis - and one well worth reading not merely for its insight into the potential of a Silicon Valley business model for the next generation media business.

While we differ with those who characterize the profession of intelligence as a specialized form of journalism or (albeit wonkish) media, we cannot argue with many of the very real parallels between intelligence community production processes and consumer outcomes issues to those experienced in the “Industry”. Among the similarities we will concede are the eternal quest for consumer attention, the need for a perceived return on investment of that time for the consumer, and the increasing demands of a more mobile consumer base more closely attuned to 24/7 information streams, and the increasing availability of numerous alternative sources of substituting products. We also see the same kinds of monolithic industrial age structures – adapted for the demands of an earlier age, and often adrift in the current environment.

Thus Andreessen’s piece strikes a unique sort of chord as we contemplate the vast legions of line analysts and field collectors who toil under cumbersome layers of management bureaucracy – both within government and their contractor counterparts. Increasingly, the barriers to entry for those attempting to produce high quality finished intelligence based on unique information sources not commonly available are falling ever faster. In many cases, only the inertia of the incumbents and the market-distorting effects of a cumbersome clearance process are arguably the only reasons why alternative products based on open source information and other, non-governmental intelligence efforts do not surface to defeat entrenched but uncompetitive offerings. But the incentives for the development of alternative models are clearly also present. And in key emerging issue areas, such as the cyber domain, there is the earliest indicators that such alternative may yet develop – perhaps paving the way for other efforts targeting other key accounts.

There is also a line that we note particularly well in the context of our earlier examination of the numbers of intelligence students graduating from the academic programs that will likely never hold a clearance, but will still seek employment utilizing their professional skillset and education. “after all, if you really can't work for the Man, why not start your own company, if you can”? We have no doubt that many of those not in the favored 28% will also take note – and we would not be willing to bet against their chances of success.

Enigmatic biographies of the damned

Via the Economist this week, we learn of the death of an adversary whose kind has nearly been forgotten. Khun Sa was a warlord who amassed a private army and smuggling operation which dominated Asian heroin trafficking from remotest Burma over the course of nearly two decades. In the end, despite indictment in US courts, the politics of a failed state permitted him to retire as an investor and business figure, and to die peacefully in his own bed.

The stories of men such as these however shaped more than a region. They are the defining features of the flow of events in a world of dark globalization. Yet these are not the biographies that are taught in international relations academia, nor even in their counterpart intelligence studies classrooms. The psychology of such men, and the personal and organizational decision-making processes of the non-state groups which amassed power to rival a princeling of Renaissance Europe, are equally as worthy of study both for historical reasons as well as for the lessons they teach about the nature of empowered individuals.

Prospective human factors and leadership analysts are not the only students which would benefit from a deeper pol/mil study of the dynamics of warlords and their followers in the Shan and Wa states. The structures which were left behind upon Khun Sa’s surrender were no doubt of enduring value to the ruling junta, and tracing the hostile connectivity provided to a dictatorial government by robust transnational organized crime is an excellent example of the kombinat model in a unique context outside of the classic Russian cases.

It is our profound hope that the study of such adversaries may also provided needed depth and contrast to the study of historical intelligence reporting regarding trafficking throughout the Golden Triangle, and the wider issues of Asian conflict dynamics, during the turbulent decades of the 70’s, 80’s, and early 90’s. It is rare to look deeply into the histories recounted by one’s adversaries in the non-state realm. But these were the fights that laid the evolutionary basis for the evolution of much harder non-state problems, and were waged on our side for the most part by organizations that were largely divorced from the classic core of the intelligence community for a very long time. These activities are no less worthy of examination, and the lessons learned from those struggles no less important to the teaching of future generations of intelligence professionals than any of the major cases of WWII and the Cold War.

Understanding intelligence consumers

One of the persistent myths that we find too often in intelligence shops – particularly among editorial and production management staff – is the idea that an intelligence consumer cannot internalize complex concepts and ideas, and thus requires extreme levels of simplification. Where exactly this myth first originated is lost to time, but that it continues to dominate the thinking of those who ought to know better is perplexing to say the least.

There is a vast difference between simplicity of expression and complexity of thought, and all too often the search for the former renders incomprehensible the latter. One sees this especially in products which emphasize the removal of common terms of art – despite the utility such short phrases bring to both consumers and producers when referencing a body of other works. The use of such terminology does not excuse the author from placing them into a relevant context, and using such context carefully within the structure of the products argument. However, all too often editors and others involved in coordination reject such expressions of brevity with blanket dismissal, citing perceived military origins of a phrase in many cases (as much of intelligence’s activities derive from war fighting origins, no matter what the current application), or as “excessively” technical. This is a grave disserve to the reader, who if they are not acquainted with a key concept associated with an intelligence issue, should be introduced to that background rather than left to assume understanding of a distorted turn of the phrase.

More and more, the results of this tired old trope become a primary reason why many consumers cite finished intelligence as irrelevant to their actual responsibilities – all the moreso when those consumers hail from segments of the community’s readership that may typically not have previously relied upon finished intelligence as part of their decision-making processes. Given the increasing levels of education demanded of even middle management, let alone the kind of experiences a typical senior might lay claim to, it is understandable that they would reject any attempt at intellectually coddling at the same time they are expected to be informed by a product. After all, it is not uncommon to be briefing or writing for a consumer with post-graduate education and longstanding experience grappling with difficult legal, business, and public policy issues. And given the number of lawyers that now make up the ranks of senior government and industry leaders – it is difficult to insist that an individual who might in another context be expected to rapidly skim and comprehend the most challenging of legal briefs must somehow never be troubled with a product written at a level higher than the typical newspaper’s 8th grade education standard.

This also contributes, in our view, to the reasons why consumers typically emphasize the value of many forms of “raw” and other internal intelligence reporting. These latter products tend to be authored for the primary readership of other intelligence professionals, and is therefore not “simplified” in the fashion one comes to expect from the editing and production process. It is no wonder then that many consumers – in some cases now more experienced than the analysts and other staff which serves them – prefer the “roughs”.

In the end, the manner in which a finished product is best crafted demands that the author understand the reader – and not simply rely on old maxims of some journalism editor’s style guide, applied without real consideration.

Alternative DOCEX models

We have frequently mentioned our aversion to involving lawyers in intelligence analysis in these pages. It is our strong belief – confirmed by countless personal experiences as well as professional anecdotes and case studies provided by others – that the current model of legal education irreversibly introduces a mental model and mindset which is inimical to the successful practice of intelligence as a profession – particularly any form of creative, strategic intelligence against non-traditional targets.

However, we unfortunately find that more and more young lawyers are entering the field. The overabundance of new graduates from an increasing number of law schools has produced more candidates than the traditional job sources within the legal profession can provide for. These graduates typically carry with them a crushing level of school loan debt, and to the inexperienced HR type, look like they have overlapping skills in the areas of research, argumentation, and writing. Despite surface appearances, the model of legal argumentation – oriented around shaping perceptions for advocacy in an adversarial setting - is profoundly different from that required to penetrate mysteries (as opposed to solving puzzles), and to piece together contradictory and constantly changing raw information into a coherent narrative designed to inform decision-maker options rather than persuade to a specific course of action.

This does not mean we are unwilling to keep tabs on what the folks in the legal field are up to, however. Via the fine folks at Volokh Conspiracy, we note an interesting story in the Washington City Paper regarding another higher order effect of the current glut of attorneys – legal temping. What interests us is not the tale of woe relayed by the newly minted and underemployed members of the Bar, but rather the manner in which the profession has sought to solve a problem which is in a very rough way equivalent to the document exploitation challenges faced by the IC.

DOCEX (now DOMEX) has long languished as a less than priority focus despite its vital contributions to key accounts. Our colleague Michael Tanji has written previously on potential for new collaborative models to attack large DOCEX / DOMEX tasks, and the resistance to those models that has been encountered even in theoretical discussion alone. No matter whether one agrees with the Army of Analysts model or not, DOCEX is an area deeply in need of the application new tradecraft and new methodologies – enabled by technology more current than the field driven database designs that have characterized most exploitation software to date. It says something profound – and none of it good – when it is a law enforcement support organization that can lay claim to what may be the most sophisticated standing DOCEX capability in the United States (though sophisticated only by comparison, and never mind about the hard target language requirements…)

We do think that the large scale legal discovery processes deserve a second look for potential adaptation by those engaged in DOCEX/DOMEX tasks. This is not because we wish to involve more lawyers in the community, but rather because there has to be a better way of organizing the kind of labour that these tasks require. While we are leery of any effort that seeks to create a machine by process, we are always willing to learn about methodologies and approaches that might better serve the task at hand.

Implications of publishing open source IMINT analysis

Some time ago, we added the excellent IMINT & Analysis blog to our sidebar for further watching. The effort is a superb example of the quality of contributions that can be made from unclassified, open source research through the application of basic analytic tradecraft. It pains us that this was not more routinely done by others in the space years ago. The technologies have been in place for some time – all that was lacking was the right mind and a willingness to devote the energies such a project would require.

To be sure, individual findings from selected research – such as Chechen urban battle damage assessments, PRC submarine activities, and the recent destruction of the Syrian suspected nuclear facility have all been chronicled using imagery. But the systematic assessment of multiple imagery sets in order to produce and publish something akin to finished geospatial intelligence is another matter entirely. Frankly, all that is lacking is cross reference to BE numbers, else one could easily mistake this as the output of a government system stripped of its markings and logos.

On the one hand, this is an unprecedented teaching opportunity for new analysts coming out of the academic side of the house. Geospatial intelligence has long been among the hardest of disciplines to inculcate within the student cohort, and many promising analysts have no doubt been steered in other directions simply by the lack of availability of resources and expertise to inspire them to pursue the path.

However, the wider availability of such products does raise concerns regarding the potential to feed adversary denial and deception efforts. High resolution commercial imagery has long carried the risk that adversaries without access to, or understanding of, the true nature of imagery collection would evolve more rapidly means to defeat such systems based on new commercial products. These risks are magnified when one can begin to glimpse the thought processes and tradecraft applied to imagery intelligence problems over the course of the production cycle.

We believe that the publishing of these kinds of finished products on an open blog such as IMINT and Analysis is not itself the problem. The real challenge arises out of the changing nature of the information environment itself. When the potential for such developments exist, they will inevitably arise in one form or another. It is better to consider their implications from a perspective of open discussion, than to attempt to second guess the effects of such activities in another context – for example perhaps that of an adversary’s intelligence service’s open source unit conducting their own version of a Red Cell assessment.

There will never be another day in the 21st century when the adversary will have less access to what was once the most sophisticated of the 20th century intelligence technical collection techniques than they do at present. The means by which such technical means may be defeated will only be easier as time passes, technology grows more common and less expensive, and the understanding of these systems from civil applications and open sources grows more sophisticated. The denial and deception problem - applied in the context of national technical means - will only get harder. The challenge to the Intelligence Community is to accelerate the pace of innovation in order to overcome the adversary’s attempts no matter how much easier the deceiver’s task may become in the future. Towards that end, we strongly suspect that the robust discussion of analytic tradecraft – including counter-deception – will do more to advance that innovation within the intelligence profession than in a closed and narrow conversation from a limited range of perspectives.

Virtual talking heads

The circulation of the survey instrument for Mercyhurst’s new research study on video format delivery of finished intelligence has occasioned no small measure of debate among participating professionals within the community. While we are glad to see the research being conducted to advance the understanding of the field, we have some serious questions about the wider implications of the use of video for product delivery. Its historical applications have occasionally resulted in unique value – particularly for certain high level consumers, such as its famous use as part of President Reagan’s daily briefing. However, for routine production it imposes burdens on development, editing and coordination, archival processes, and technology / connectivity requirements that create issues far beyond the initial considerations of communications efficacy.

The entire discussion also reminds us of the earlier attempts to re-create intelligence briefings into virtual formats through automated processes. First popularized in response to Web 1.0 start-ups such as the virtual news readers such as Anna Nova, the concept never moved beyond research and development phases precisely for some of the same reasons. The removal of the human briefing element also created a disconnect between attention and pacing, which alienated many consumers whose areas of interest diverged from the set piece delivery – but who may have been more productively engaged by a dynamic briefer. There is also the subtle but very real impact of the uncanny valley – the noticeable aspects of the artificial in “too human” like virtual entities. (We wish to note one of our close colleagues’ wise but whimsical suggestion to replace a human virtual briefer with an object of classic children’s theatre. If nothing else, we would love to have such a product on hand to inject a bit of levity into meetings in which the participants take themselves too seriously.)

Set piece delivery tends to work best in environments where the consumer can devote only partial attention to the product – such as audio delivery of briefing or lecture materials, designed for those who are looking for cognitive engagement during other routine tasks such as driving, physical exercise, or the like. We have not yet seen an environment in which video delivery could meet this same niche – particularly for products which demand a higher degree of attention than watching CNN with the sound turned down in the typical watch center fashion. Virtual talking heads suffered the same problem.

There also remains the serious question of the longer term impacts of removing one of the few opportunities for direct interaction between consumers and intelligence professionals, where the intangibles of human relationships are nowhere as easily quantified. Perhaps there are some aspects of intelligence that simply ought not be automated, to the ultimate benefit of consumer outcomes.

Nonetheless, we will be interested to see the results of the study, and happy to see the literature of intelligence advanced in this area. We can only hope other academic studies will continue to build understanding of the nature of intelligence communication.

Home-brewed, open source cryptanalysis

Among the odder developments created by the inevitable trends in Moore’s Law of increasing computing power (at ever cheaper price points) has been the feasibility of private cryptanalysis capabilities functioning at effectiveness which only a few years ago would likely have been possible solely with the resources of the nation state.

These bootstrapped rigs as a rule tend to emphasize the lowest possible cost configurations – a natural consideration given that most are assembled on a shoestring budget by university researchers or other computer sciences academic types. They have been assembled for diverse purposes of privacy advocacy, systems research, and some simply for the sheer technical interest of the thing.

We particularly like the COPACOBANA system, a FGPA based parallel computing design optimized to attack symmetric ciphers, created entirely using commercial off the shelf components. The system can typically identify DES keys within less than a week of effort, at a cost of about $10,000 per machine. The now obsolete Data Encryption Standard cipher was a widely used algorithm between its approval in 1977 and its withdrawal in 2002, and remains in use in some legacy systems in the commercial environment to this day. A stronger derivative algorithm, 3DES, also remains more commonly in use in some applications, with end of life projected to 2030. DES did have a good long run, having survived the public disclosure of the previously secret technique of differential cryptanalysis to which it might have been vulnerable, had NSA and IBM not supposedly collaborated on a stronger implementation during its development phases. An earlier custom built FGPA rig - DeepCrack – was built to attack the cipher successfully, but at a cost of nearly a quarter million dollars.

These cryptanalysis systems may be jury rigged, but they are undeniably effective – and cheap. And more powerful implementations are no doubt easily within the budgets of smaller nation-states, if not wealthy non-state actors.

However, it is the development of the alternative track of home-brewed cryptanalysis that gives us pause. One competitor in the attack against the DES cryptosystem was a distributed computing network that relied on the contributions of unused capacity from volunteers’ personal boxen. The DESCHALL project achieved its successful break using 78,000 contributors over the course of three months. The project demonstrated the architecture – more advanced attacks are mostly a matter of optimization and scale.

It is from this that we begin to ponder our greater concerns. The development of very large scale botnets, made up of aggregated collections of individual systems compromised by malware, offer far greater potential computing capacity – both an individual basis per processor as well as in overall numbers of contributing systems. The widely discussed STORM botnet remains perhaps the largest of such malicious aggregates that has publicly identified. While exact numbers remain subject to debate, STORM is believed to consist of up to 50 million individual systems. And while recent indicators are that the botnet is being sub-divided into smaller segments for illicit commercial sale, this kind of capacity in the hands of non-state actors is truly revolutionary. Its potential applications to home-brewed cryptanalysis are clear.

Interestingly enough, the STORM botnet also appears to itself utilize encryption in order to secure its own control communications, and to permit more effective illicit commercialization. (Although surprisingly not a GOST standard cipher). The use of STORM, or the application of similar very large scale capabilities, to the flip side of the cryptographic equation is surely not far off.

If nothing else, it is good reason to accelerate those academic intelligence studies that might have an interest in cryptanalysis.

Lost intelligence history

We have recently had reason to ponder the side effects of the generation shift currently underway within the intelligence profession. The course of that discussion focused on capturing the tactic knowledge and life experiences of the intelligence professionals about to retire. There is a great need to transfer that accumulated body of wisdom as best as possible to the next generation, given the lack of continuity created by the short sighted and disastrous hiring gaps of the 1990’s.

But the discussion also raised another thought – what about the intelligence history that is now occurring, often in the hands of an entirely new generation, in those places distant from the traditional core of the community?

There are ever larger numbers of small units, task forces, and cells operating out in the hinterlands in the forward expeditionary environments. They are staffed with only a few billets, and their lack of formal training and connectivity has long been a subject of debate. But there is a flipside to that disconnect, not merely in its effects on the performance of their positions, but also on the propagation of their innovations and experiences to the wider community. While many of these places may simply be re-inventing the wheel, it is equally as likely that new approaches and unique answers may emerge from the diversity of talent, backgrounds, and targets that is out there in the world beyond the traditional major agencies.

Likewise, the privatization of intelligence continues to demonstrate its impact – but rarely are these issues examined in the light of history’s perspectives. Reorganizations far less significant feature prominently in the history of many agencies, but a change in contractors that may result in an order of magnitude larger shift in assignments goes without mention. We might uncharitably believe that this lack of attention arises because many in government wish the complex issue would simple go away and not trouble their thoughts, as they ponder the uncomfortable potential of a future driven by real competition. However, it is equally as likely that these are issues that simply have not been fully considered by those engaged in the study of intelligence history. What few efforts we are aware of to contemplate the dimensions of the privatized efforts are typically done as part of internal capital building within the better contractor shops themselves – frequently on overhead internal billing or as volunteer efforts.

The lessons learned from the far flung places in which the intelligence enterprise now finds itself cannot be allowed to slip away without notice. The solutions to current problems – successful or otherwise – are equally if not more important than the oral history of the profession’s previous generation. It takes a different sort of mind, and a different style of historian, to approach the young 20-somethings out there in the disconnected wilds, but what one might find there is no doubt surprising and intriguing well in its own right. After all, many of these young professionals may have two or three tours under their belt already, and are the embodiment of the Long War. Their deeds deserve a record, and that record may hopefully inspire the kind of self-reflective examination that is critical to advancing the study of the intelligence.