The privatization of intelligence history

The preservation of iconic history is one of the most important cultural and institutional tasks that the intelligence community can perform to ensure the continued relevance of its traditions as an intellectual pursuit among the generations of professions which follow. A shadowed profession needs more than most the few tangible symbols of what it is we stand for, what we have accomplished, and what we ought to emulate – if not in strict form or function, than in spirit and ideal. It is these few tokens (and their stories) – whether the odd item somehow passed down from those that were there, or the unique place which by virtue of the accidents of geography and function became key to a major program or structure – that also help to cement a shared vision of an increasingly distributed profession.

Many IC leaders agree to this principle in theory. Yet when the heart of the World War II cryptanalytic effort at Bletchley Park was left to decay, the international intelligence community of the Allied powers paid little attention. Of course, this is not a new problem, but efforts to preserve the history of intelligence have ranked low on the priority list in the face of unprecedented wartime demands coupled with the critical need to re-capitalized aging operational infrastructure neglected during the lean years of the 1990’s. And while some might say this is strictly a British problem, the long history of the special relationship – and particularly the key role played by shared signals intelligence efforts in creating that relationship – dictates American concern (and like concern for the rest of the Five Eyes partners).

Thus we find privatization emerging in a new and unexpected manner. In this case, it is a fundraising effort led by the IBM and PGP corporations, designed to remedy with private donations the gaps left by government abandonment. This is a development which resonates on multiple levels.

PGP as a firm arguably exists as a privatized solution to another government shortfall – the need to protect sensitive corporate and critical infrastructure communications from unauthorized intercept. In the early days of the Cold War, this was considered an inherently governmental responsibility – and one that early cryptographic policy reserved only for an exceptionally small segment of the corporate world, usually only directly associated with a highly limited number of defense industrial base or Federal level financial institutions. This deliberate omission of protection for the vast engine of much of the countries economy led to innovation and the re-birth of an entire commercial market. For like in many aspects of intelligence, the state monopoly of the WWII and early Cold War era was a historical anomaly. Commercial codes had long been in use for protection of sensitive international business communications. Yet the advent of professional cryptanalysis organizations – and the computing resources that they developed to aid them in their tasks - would destroy most pre-war systems based on too simple substitutions or primitive algorithms.

While the techniques of public key encryption by which PGP became the world’s standard for commercial communications security were indeed invented anew without prior knowledge of government activities in the area, it is now well documented that these techniques originated much earlier in the darkest corners of the intelligence establishment. However Non-Secret Encryption, as it was then called, merely serves to illustrate the gap between protection offered to the public versus the private sector. (One can make allowances, however, for the desire to keep all aspects of technology possibly used to secure nuclear weapons Permissive Action Links entirely out of the public view in any form. But thus has the world changed now.)

While contemporary industry's interest in Station X is no doubt driven more by the history of computing itself than the preservation of an intelligence icon, it is fitting to see the structures which emerged from the Black Valley step up to ensure that this monument will continue to endure - especially given that no such symbols remain of their own struggles.

h/t Slashdot

Living intelligence history through hobbyist cryptanalysis

Following up with the latest in the recent stream of news regarding the ever so eccentric world of code breaking, we are both impressed and amused with the effort at Bletchly Park to recreate the machines that automated much of the attacks against German cryptosystems – and arguably shortened WWII by an incalculable duration. The reconstructed Colossus will test its mettle once again versus the Lorenz cipher, code named FISH.

The original FISH break came about in no small part due to operator error - which involved retransmission of the same lengthy message twice, using only slightly modified cipher settings. Let us hope that similar fortune favours the boffins of Station X once again.

Given that the recreated test also involves a comparison against a modern virtual emulation of the Colossus system, we cannot help but wonder what the results might be if a modern botnet-based cryptanalytic attack was to be added to the race? To be fair, we suppose a virtual analogue of the Y Service intercept system would also have to be crafted – perhaps paired with a software defined radio link or a TCP/IP intercept capability of some flavour. Of course, we would rather prefer the pretty young female clerks that used to operate the radio sets...


h/t Futurismic


UPDATE:

The rebuilt Colossus worked as expected, and achieved a successful break of the encrypted transmissions in a little over three hours. A modern computer analogue did nonetheless manage a successful break faster. Joachim Scheuth, a German cryptologic hobbyist, is to collect the first round of drinks at Bletchley.

However, the exercise did demonstrate the sheer difficulty of SIGINT – particularly in the interception of the weak radio signals of the day using contemporaneous equipment. Something to ponder in this age of far more sensitive systems, and far more elevated expecations.

Home-brewed, open source cryptanalysis

Among the odder developments created by the inevitable trends in Moore’s Law of increasing computing power (at ever cheaper price points) has been the feasibility of private cryptanalysis capabilities functioning at effectiveness which only a few years ago would likely have been possible solely with the resources of the nation state.

These bootstrapped rigs as a rule tend to emphasize the lowest possible cost configurations – a natural consideration given that most are assembled on a shoestring budget by university researchers or other computer sciences academic types. They have been assembled for diverse purposes of privacy advocacy, systems research, and some simply for the sheer technical interest of the thing.

We particularly like the COPACOBANA system, a FGPA based parallel computing design optimized to attack symmetric ciphers, created entirely using commercial off the shelf components. The system can typically identify DES keys within less than a week of effort, at a cost of about $10,000 per machine. The now obsolete Data Encryption Standard cipher was a widely used algorithm between its approval in 1977 and its withdrawal in 2002, and remains in use in some legacy systems in the commercial environment to this day. A stronger derivative algorithm, 3DES, also remains more commonly in use in some applications, with end of life projected to 2030. DES did have a good long run, having survived the public disclosure of the previously secret technique of differential cryptanalysis to which it might have been vulnerable, had NSA and IBM not supposedly collaborated on a stronger implementation during its development phases. An earlier custom built FGPA rig - DeepCrack – was built to attack the cipher successfully, but at a cost of nearly a quarter million dollars.

These cryptanalysis systems may be jury rigged, but they are undeniably effective – and cheap. And more powerful implementations are no doubt easily within the budgets of smaller nation-states, if not wealthy non-state actors.

However, it is the development of the alternative track of home-brewed cryptanalysis that gives us pause. One competitor in the attack against the DES cryptosystem was a distributed computing network that relied on the contributions of unused capacity from volunteers’ personal boxen. The DESCHALL project achieved its successful break using 78,000 contributors over the course of three months. The project demonstrated the architecture – more advanced attacks are mostly a matter of optimization and scale.

It is from this that we begin to ponder our greater concerns. The development of very large scale botnets, made up of aggregated collections of individual systems compromised by malware, offer far greater potential computing capacity – both an individual basis per processor as well as in overall numbers of contributing systems. The widely discussed STORM botnet remains perhaps the largest of such malicious aggregates that has publicly identified. While exact numbers remain subject to debate, STORM is believed to consist of up to 50 million individual systems. And while recent indicators are that the botnet is being sub-divided into smaller segments for illicit commercial sale, this kind of capacity in the hands of non-state actors is truly revolutionary. Its potential applications to home-brewed cryptanalysis are clear.

Interestingly enough, the STORM botnet also appears to itself utilize encryption in order to secure its own control communications, and to permit more effective illicit commercialization. (Although surprisingly not a GOST standard cipher). The use of STORM, or the application of similar very large scale capabilities, to the flip side of the cryptographic equation is surely not far off.

If nothing else, it is good reason to accelerate those academic intelligence studies that might have an interest in cryptanalysis.

The relics of the code wars past

If you are looking for that perfect gift for the spook who has everything, you could do far worse than to get them their own fully functional authentic Engima cipher machine.

There are few items more totemic in their physical incarnation than this humble collection of wires, rotors, and lights. Yet this machine, and the false confidence in its security, killed more Germans than all of Bomber Commands - and in its closely related incarnations, killed more Japanese than the atomic bomb.

It is also the symbol of perhaps the greatest victory of intelligence in the history of the Western services. Nothing like it may ever exist again as a focal point of such unique and incomparable import and value.

We are a few days late in catching this item, so bids have now exceeded $28,000 (after delisting and re-listing of the auction. (If you simply must have it for yourself, it's an immediate purchase price of $50,000). But for those that can afford to give that special someone in their life the very best, it is more than worth it.

h/t Castle Arrgh!, Slashdot, and the many others which noticed this most unusual of listings.

Historical perspectives on military and intelligence privatization – codes and ciphers in China’s warlord era

Intelligence as an organized activity in the modern era began as an amateur’s game, made more serious by the deadly business of war throughout the 20th century. And while much attention has been paid to the evolution of the early effort of pioneers which became the basis for the major nation-state programs of the major conflicts that convulsed the globe in the three world wars of the last century, we occasionally have reason to be reminded of, and to reflect upon, the efforts which the intelligence studies history has nearly forgotten.

Among those are an interplay of requirements, technology, and actors which resulted in one of the most interesting artifacts we have ever had privilege to lay eyes on – one of the earliest codebooks used by a private military company. (An item that we had hoped one day to be graced to acquire, but unfortunately as the years have passed it seems that is not to be.)

The nearly continuous conflicts of the Chinese warlord period resulted in the development of a major market for mercenaries and arms merchants engaged by the various factions to support and supply their forces. Communications of the day were primitive, but the telegraph had made ingress to the country and represented the only effective link between the forward representatives of the private military companies of the day (often individual entrepreneurs and adventurers) and their larger networks of investors, suppliers, and strategic level intelligence (usually provided solely through open sources and RUMINT).

The security of these communications were however a paramount consideration in the ever shifting political currents of an intensely factionalized country. Thus, a variety of code systems and ciphers came to be employed by those smart enough, or experienced enough, to pay attention to such matters.

The technology of the day required almost all messages to be sent via Morse, and messages would pass through any number of hands during transmission, receipt, and delivery. Further, the system was a heavily Anglicized development which did not handle Chinese characters well, although a numeric transliteration system was in use. Additionally, major companies (especially in the banking and shipping industries) had adopted a series of brevity codes in order to save telegraph transmission costs when handling routine message traffic (although not without push-back from the major telegraph providers, who saw lost business opportunities in the shorter messages and increased costs due to the greater difficulty and greater error rates which were incurred in sending enciphered text – the net neutrality battles of the Victorian Internet).

It is out of those commercial codes that the early efforts arose to provide for secure communications amongst those in the private military industry of the day. A number of firms published primitive codebooks for both brevity and security purposes - a commercial market for cryptology which would preserve elements of the nascent American signals intelligence establishment during the turbulent days following World War I. Many of these codebooks were adopted throughout the private military industry in China, and evolved in modified fashion throughout the course of operations. In one surviving example which we are familiar with, the phrase tables had been extended to cover a wide range of situations the original designers had no doubt never envisioned. The sometimes hastily scrawled handwritten notes of nearly a hundred years ago are still quite familiar to anyone that has ever been in the field, attempting to obtain logistics support or to acquire new munitions and matériel.

The 21st century has seen the world of cryptography come full circle. Now, it is again commercial technology - commonly available at the individual level through public key and elliptic curve encryption as well as widely published, mathematically robust algorithms such as AES / Rijndael - that is featuring prominently in the forward theatres of the Long War, where the private military industry finds its most active current markets and the field of communications security has some of its most interesting opportunities for substantial new contributions.

We would wish to see this sort of history better explored by the academics of the intelligence studies field, and the longstanding nature of the private military enterprise taken into account during the all too often overwrought discussions around the privatization of intelligence and of warfare. The literature of intelligence, and of national and international security studies, would no doubt greatly benefit - and who know what young minds might thus be inspired to accomplish?