Technical OSINT innovation contest: the 2008 Malware Challenge

While the worlds of most OSINT analysts do not typically overlap with those working in the more rarified fields of digital network intelligence, forensic analysis, and network warfare, there are a highly specialized subset that may be interested in testing their skills as part of a challenge of their own. While clearly not as high profile as the recent DNI OSINT contest, the 2008 Malware Challenge promises interesting responses of its own.

The winners of the malware challenge will be announced at the 2008 Ohio Information Security Summit on 31 October 2008. We had not previously seen this conference, but it appears to be a small regional conference that is unusually well attended by the usual round of ex-spooks and ex-cops that have moved into the cyber security industry as of late.

The challenge scenario is reproduced below:

"A system administrator within your organization has come to you because a user's PC was infected with malware. Unfortunately, anti-virus is unable to remove the malware. However, the administrator was able to recover the suspected malware executable. Your job is to analyze the malware.
Participants should download the malware sample and analyze it. The end result should be a document containing details on the analysis performed. The analysis document can be written in any form, but the questions and statements below should be answered within it. Participants should note what questions are being answered.
The questions...
* Describe your malware lab.
* What information can you gather about the malware without executing it?
* Is the malware packed? If so, how did you determine what it was?
* Describe the malware's behavior. What files does it drop? What registry keys does it create and/or modify? What network connections does it create? How does it auto-start, etc?
* What type of command and control server does the malware use? Describe the server and interface this malware uses as well as the domains and URLs accessed by the malware.
* What commands are present within the malware and what do they do? If possible, take control of the malware and run some of these commands, documenting how you did it.
* How would you classify this malware? Why?
* What do you think the purpose of this malware is?

Bonus questions: (These questions are not required to be answered but could be used to break a tie for prizes.)
* Is it possible to find the malware's source code? If so, how did you do it?
* How would you write a custom detection and removal tool to determine if the malware is present on the system and remove it?

Analysis documents should be submitted in PDF format to 2008challenge@malwarechallenge.info by 12:00 Midnight EST (5:00 AM GMT) on October 26, 2008."

Additional information, including other contest rules and FAQ, can be found at the challenge website.

Interestingly, we note that Steve Jackson Games is among the sponsors providing prizes for the winners. SJG was most famously the victim of a botched Secret Service raid in 1990, which seized files and texts that were part of one its published gaming lines. For those that are not familiar with this disastrous episode from the earliest days of the cyber intelligence account, it was best recounted in Bruce Sterling’s still timeless book, The Hacker Crackdown. (In our opinion, this is also a text which should be mandatory reading for those involved in SIGINT, MEDEX, or eCrime analysis. And while the USSS has indeed come a long way since then, we do from time to time encounter other shops still grappling to come to terms with the new threat environment with often equally absurd results.)


h/t Spy Logic

A glimpse of a future naval (and network) special operations mission: Google edition

The jesters and the futurists have long featured offshore structures as the future of human activities. Reality continues to bear out these predictions. We have previously discussed large scale offshore habitation structures and their potential impact for future intelligence and unconventional warfare problems. This time around, it is the concept of the maritime data center – previously discussed as a moored vessel – recast as an offshore terminal type platform.

We are enthused by the possibility of seeing a real life data haven come into its own – and run by a professional multinational entity (rather than the sad sort of anarchist disgrace that was Sealand). But one can easily see the fascinating potential for convergence of a whole range of threats directed against such facilities, involving both naval and network conflicts.

Once again, our froggy friends will no doubt be forced to ponder such actions. And while there are no doubt legions of would-be hackers just waiting to strap on a wetsuit to live out their own episode of the Rock against a network contagion, they are apt to be disappointed. One suspects that the signals intelligence folks will be far more likely to have to come to terms with what the widespread proliferation of such offshore datacenter platforms will mean in an environment where bandwidth and storage are nearly infinite and entirely cheap, and may be rented from the cloud through a complex and shift network of shell companies, taking advantage of low staffing levels and limited oversight incentives. The Russian Business Network’s latter day successors will no doubt be wet.

h/t Futurismic

The literature of cryptologic intelligence, declassified

Whole volumes of the intelligence literature produced within one of the oldest of the nation’s organizations are too often ignored by academics and other practitioners outside of the narrow field of signals intelligence. The literature of the technical side of the house tends to be exceptionally arcane – even by the eccentric standards of the rest of the IC – littered with mathematics, circuit diagrams, and radio propagation sketches. Its historical materials assume a familiarity that many outsiders cannot broach. And above all, this literature remains exceptionally protected, often for decades longer than counterpart publications in other agencies.
But the reader would be remiss to ignore the recently declassified series of articles taken from the various in-house journals and historical studies of our friends at Fort Meade. While still a small collection, it is a remarkable aggregation of materials of great potential value for those engaged in the hard task of teaching intelligence to the next generation of budding young professionals.

We are particularly hopeful that this round of declassifications will also spark a new interest in the history and theory of this venerable discipline. The second wave of publications that result from such interest are often invaluable companions to the primary works authored by practitioners, as academics attempt to interpret and re-contextualize material which has often long ago passed into the domain of the unexamined assumptions of common knowledge.

So hie thee hence, dear readers, and we hope to see new life emerge in the commentary and discussion of these thoughts of those that came before, and who laid the foundation for the practice of the art and science as we now know it.

Intelligence history in the Black Valley

We are quite fortunate to number among our readers a gentleman of expertise and innovation, who passed along an excellent video in which he can be seen here lecturing at Google's occasional talks series. The topic of conversation was what we would consider the early history of intelligence privatization – the development of the early partnerships to pursue new scientific and technical intelligence operations against Germany and the Soviet Union. For those generally unfamiliar with the importance of the Wizard’s War, the lecture also serves as an excellent introduction to basic electronic warfare concepts and the SIGINT / ELINT challenges required to effectively support EW during WWII and the Cold War era.

The lecture in particular resonates as it tracks the eventual higher order effects of the early Cold War surge in a way that is rarely done in intelligence history surveys. One can almost for a moment imagine a different lecturer, sitting in a room somewhere in a start-up turned world changing firm of not so distant futurity, exploring the impact of innovations in the early years of the Long War from a similar perspective and style. In our mind’s eye, we perhaps think that such a future lecture might focus on the IED Defeat fight, or perhaps the manhunting problem – but of course it has not been given to us to seen the future, only merely to glimpse the outlines of its potential.

For now, we are pleased to see this kind of history surfacing, and being spread in places where creative and imaginative young minds cluster, especially by such a gifted speaker and guide. Let us hope that it may inspire the next generation of intelligence developments and successes, even as it enlightens us to the pasts we share.

SIGINT in the exaflood environment

There has been a lot of talk recently regarding the implications of the rising rate of data exchange for policy issues such as network neutrality and broadband penetration. The term exaflood - coined by one particularly lobbying group - is apt enough, even if one doesn’t necessarily agree with their proposed solution approaches.

There has been however little discussion of the implications of this expansion for the intelligence community – at least in public and academic circles. The current debate is too much caught up in the things of lawyers and politicians, wrangling continuously over legislation drafted when networks were switched by telephone system hardware and connectivity not much different from the transoceanic telegraph systems of a century previously. Those are questions of whether or not the US should even be engaged in such activities, and are far too closely reminiscent of the short-sighted political decisions which resulted in the closure of the Black Chamber – with the potential for equally devastating consequences.

Traditional SIGINT techniques – even within the relatively new realm of digital network intelligence – are the products of an earlier era, in which the target set and its emanations were distinct enough from its environment to be amenable to capture and analysis using a certain degree of discrimination. The kinds of intelligence that will be required against the adversaries of tomorrow will be increasingly less able to rely on the traditional tradecraft which is undergirded by such assumptions.

We do agree with the statement, frequently attributed to former Assistant Director of Central Intelligence for Analysis & Production Mark Lowenthal, to the effect that “there is no such thing as information overload, only poor analytical strategies.” However, the exaflood will challenge both collection and analytical strategies such as never before. Against this backdrop, we look to the continuing infrastructure, language, and human resources challenges faced by those in this section of the community, and greatly wonder if our future community will be adequate to the task.

This is an intelligence challenge that goes far beyond the classic view of conversations that is implied with the COMINT model. Critical value will be found in the relationship and intentionality of information within the digital deluge as much as the actual items itself. One need only look to the importance of identifying individuals in video and photo, such as VOA Iran scandal or the as yet unresolved counterintelligence questions surrounding the relatives of Nada Prouty, for a glimpse of the future impact of such changes on the way the community does business.

Short of some unexpected development in artificial intelligence which will allow for a weakly-godlike appreciation of this overwhelming mass of largely undifferentiated packets, new approaches will be desperately needed. One can only get so far with human-centered processing strategies (at least without introducing a much large pool of vetted practitioners in a Mechanical Turk like system.) And we will still face the essential limitations of time – the time needed to immerse, incubation, and cultivate a longer term appreciation of the narrow windows into complex issues SIGINT will provide, even in the new environment.

It is our contention that the native competency of intelligence in the cyber environment has yet to be recognized. The nature of cyber intelligence in its mature form will have shifted so far from its roots in the SIGINT and even OSINT disciplines that it may well be unrecognizable to the practitioners of today (no matter how complex the Large Scale Internet Exploitation System or how many dark web projects are built for OSINT purposes).

We do not yet know what the new discipline will look like, nor how it will ultimately be shaped. But we are very interested to observe its evolution, and are privileged to be present at its creation.

Living intelligence history through hobbyist cryptanalysis

Following up with the latest in the recent stream of news regarding the ever so eccentric world of code breaking, we are both impressed and amused with the effort at Bletchly Park to recreate the machines that automated much of the attacks against German cryptosystems – and arguably shortened WWII by an incalculable duration. The reconstructed Colossus will test its mettle once again versus the Lorenz cipher, code named FISH.

The original FISH break came about in no small part due to operator error - which involved retransmission of the same lengthy message twice, using only slightly modified cipher settings. Let us hope that similar fortune favours the boffins of Station X once again.

Given that the recreated test also involves a comparison against a modern virtual emulation of the Colossus system, we cannot help but wonder what the results might be if a modern botnet-based cryptanalytic attack was to be added to the race? To be fair, we suppose a virtual analogue of the Y Service intercept system would also have to be crafted – perhaps paired with a software defined radio link or a TCP/IP intercept capability of some flavour. Of course, we would rather prefer the pretty young female clerks that used to operate the radio sets...


h/t Futurismic


UPDATE:

The rebuilt Colossus worked as expected, and achieved a successful break of the encrypted transmissions in a little over three hours. A modern computer analogue did nonetheless manage a successful break faster. Joachim Scheuth, a German cryptologic hobbyist, is to collect the first round of drinks at Bletchley.

However, the exercise did demonstrate the sheer difficulty of SIGINT – particularly in the interception of the weak radio signals of the day using contemporaneous equipment. Something to ponder in this age of far more sensitive systems, and far more elevated expecations.

Home-brewed, open source cryptanalysis

Among the odder developments created by the inevitable trends in Moore’s Law of increasing computing power (at ever cheaper price points) has been the feasibility of private cryptanalysis capabilities functioning at effectiveness which only a few years ago would likely have been possible solely with the resources of the nation state.

These bootstrapped rigs as a rule tend to emphasize the lowest possible cost configurations – a natural consideration given that most are assembled on a shoestring budget by university researchers or other computer sciences academic types. They have been assembled for diverse purposes of privacy advocacy, systems research, and some simply for the sheer technical interest of the thing.

We particularly like the COPACOBANA system, a FGPA based parallel computing design optimized to attack symmetric ciphers, created entirely using commercial off the shelf components. The system can typically identify DES keys within less than a week of effort, at a cost of about $10,000 per machine. The now obsolete Data Encryption Standard cipher was a widely used algorithm between its approval in 1977 and its withdrawal in 2002, and remains in use in some legacy systems in the commercial environment to this day. A stronger derivative algorithm, 3DES, also remains more commonly in use in some applications, with end of life projected to 2030. DES did have a good long run, having survived the public disclosure of the previously secret technique of differential cryptanalysis to which it might have been vulnerable, had NSA and IBM not supposedly collaborated on a stronger implementation during its development phases. An earlier custom built FGPA rig - DeepCrack – was built to attack the cipher successfully, but at a cost of nearly a quarter million dollars.

These cryptanalysis systems may be jury rigged, but they are undeniably effective – and cheap. And more powerful implementations are no doubt easily within the budgets of smaller nation-states, if not wealthy non-state actors.

However, it is the development of the alternative track of home-brewed cryptanalysis that gives us pause. One competitor in the attack against the DES cryptosystem was a distributed computing network that relied on the contributions of unused capacity from volunteers’ personal boxen. The DESCHALL project achieved its successful break using 78,000 contributors over the course of three months. The project demonstrated the architecture – more advanced attacks are mostly a matter of optimization and scale.

It is from this that we begin to ponder our greater concerns. The development of very large scale botnets, made up of aggregated collections of individual systems compromised by malware, offer far greater potential computing capacity – both an individual basis per processor as well as in overall numbers of contributing systems. The widely discussed STORM botnet remains perhaps the largest of such malicious aggregates that has publicly identified. While exact numbers remain subject to debate, STORM is believed to consist of up to 50 million individual systems. And while recent indicators are that the botnet is being sub-divided into smaller segments for illicit commercial sale, this kind of capacity in the hands of non-state actors is truly revolutionary. Its potential applications to home-brewed cryptanalysis are clear.

Interestingly enough, the STORM botnet also appears to itself utilize encryption in order to secure its own control communications, and to permit more effective illicit commercialization. (Although surprisingly not a GOST standard cipher). The use of STORM, or the application of similar very large scale capabilities, to the flip side of the cryptographic equation is surely not far off.

If nothing else, it is good reason to accelerate those academic intelligence studies that might have an interest in cryptanalysis.

Legal intrusions into foreign intelligence activities

There has been a recent and disturbing trend of domestic courts increasingly intruding into the realm foreign intelligence activities, an area throughout American history which has been the sole operational province of the Executive Branch, funded and overseen by the Legislative. Whether concerning detainee operations, communications intelligence programs, or even assessments of the threat posed by illegal combatants – the dangerous idea that foreign intelligence (as an activity) should be treated by legal regimes in the same manner as information gathering in the law enforcement context.

This is a damningly difficult concept to combat. Intelligence activities should and must always take place within the framework of a rule of law, and subject to the control and oversight of the elected branches of government. Reviewing intelligence activities on a case by case basis within a judicial framework is precisely the wrong approach when attempting to maintain situational awareness at the national level in a complex and dangerous world.

In recent weeks we have been particularly troubled to see a number of items which point towards the intellectual foundations of a potentially radical expansion in the scope of judicial overreach into foreign intelligence activities. The continued political furor over the recent leaks regarding the “Terrorist Surveillance Program” has shown just how damaging such intrusions can be, and how terribly frequent the improper discussion of sensitive programs by those who so callously break their most sacred oaths. When the DNI himself indicates that the political fights and legal wrangling has had operational impact, it is more than time to worry.

Worse yet, we detect a discernable strain of legal thinking which now seeks to impose restrictions not only on the collection of information, but on its use. The idea that a warrant might be required to search against previously accumulated foreign intelligence materials sounds absurd, but recent legal opinions appear to have laid the groundwork for such an argument in future cases. This would also be very nearly absolutely fatal in the context of fusion and collaboration for homeland security intelligence purposes (particularly if critical elements of the intelligence picture are obtained from foreign intelligence activities of DOD and other agencies, as if often the case.)

We have long maintained that the mindsets of the lawyer and the intelligence professional are diametrically opposed. The first seeks to present a structured picture through adversarial argumentation, and by training attacks to exclude evidence from the picture to support a particular viewpoint. The latter struggles to understand puzzles and mysteries, and to assemble a coherent narrative in the face of incomplete, conflicting, and deceptive information in order to support the decision-maker’s choices regarding courses of action. Allowing the lawyers to dictate further the key aspects of the world of intelligence – and allowing intelligence activities to be framed into an “investigative” basis rather than continuing inquiry into matters of standing interest – will be the death of the profession.

In part, we come so near to such a state of affairs due to the lack of formalization of intelligence activity as a profession in its own right. It is too easy for those without a long term stake in the process to formulate “reform”, and in the current hyper-politicized environment, see those measures implemented without regard to their effect within the community. What few efforts we have seen to move towards professionalization have been far too much in the character of rent-seeking, and too little towards establishing an articulable and defensible basis for the actions and requirements which allow for the successful understanding of the difficult accounts and hard targets with which we are faced.

We have but a little time for the intelligence studies academia to step up and meet this challenge, before too much is lost to those who pursue studies of the field from a perspective which is distinctly alien to the history and the effective practice of the art and science of intelligence.

Historical perspectives on military and intelligence privatization – codes and ciphers in China’s warlord era

Intelligence as an organized activity in the modern era began as an amateur’s game, made more serious by the deadly business of war throughout the 20th century. And while much attention has been paid to the evolution of the early effort of pioneers which became the basis for the major nation-state programs of the major conflicts that convulsed the globe in the three world wars of the last century, we occasionally have reason to be reminded of, and to reflect upon, the efforts which the intelligence studies history has nearly forgotten.

Among those are an interplay of requirements, technology, and actors which resulted in one of the most interesting artifacts we have ever had privilege to lay eyes on – one of the earliest codebooks used by a private military company. (An item that we had hoped one day to be graced to acquire, but unfortunately as the years have passed it seems that is not to be.)

The nearly continuous conflicts of the Chinese warlord period resulted in the development of a major market for mercenaries and arms merchants engaged by the various factions to support and supply their forces. Communications of the day were primitive, but the telegraph had made ingress to the country and represented the only effective link between the forward representatives of the private military companies of the day (often individual entrepreneurs and adventurers) and their larger networks of investors, suppliers, and strategic level intelligence (usually provided solely through open sources and RUMINT).

The security of these communications were however a paramount consideration in the ever shifting political currents of an intensely factionalized country. Thus, a variety of code systems and ciphers came to be employed by those smart enough, or experienced enough, to pay attention to such matters.

The technology of the day required almost all messages to be sent via Morse, and messages would pass through any number of hands during transmission, receipt, and delivery. Further, the system was a heavily Anglicized development which did not handle Chinese characters well, although a numeric transliteration system was in use. Additionally, major companies (especially in the banking and shipping industries) had adopted a series of brevity codes in order to save telegraph transmission costs when handling routine message traffic (although not without push-back from the major telegraph providers, who saw lost business opportunities in the shorter messages and increased costs due to the greater difficulty and greater error rates which were incurred in sending enciphered text – the net neutrality battles of the Victorian Internet).

It is out of those commercial codes that the early efforts arose to provide for secure communications amongst those in the private military industry of the day. A number of firms published primitive codebooks for both brevity and security purposes - a commercial market for cryptology which would preserve elements of the nascent American signals intelligence establishment during the turbulent days following World War I. Many of these codebooks were adopted throughout the private military industry in China, and evolved in modified fashion throughout the course of operations. In one surviving example which we are familiar with, the phrase tables had been extended to cover a wide range of situations the original designers had no doubt never envisioned. The sometimes hastily scrawled handwritten notes of nearly a hundred years ago are still quite familiar to anyone that has ever been in the field, attempting to obtain logistics support or to acquire new munitions and matériel.

The 21st century has seen the world of cryptography come full circle. Now, it is again commercial technology - commonly available at the individual level through public key and elliptic curve encryption as well as widely published, mathematically robust algorithms such as AES / Rijndael - that is featuring prominently in the forward theatres of the Long War, where the private military industry finds its most active current markets and the field of communications security has some of its most interesting opportunities for substantial new contributions.

We would wish to see this sort of history better explored by the academics of the intelligence studies field, and the longstanding nature of the private military enterprise taken into account during the all too often overwrought discussions around the privatization of intelligence and of warfare. The literature of intelligence, and of national and international security studies, would no doubt greatly benefit - and who know what young minds might thus be inspired to accomplish?