/* */

12 February 2006

Information overload in the real world continued - the open source dimension

Your authors will rarely delve into discussions of meta-blogging…. Those most self-referential of conversations regarding the sausage making which produces words for screen. It is our considered opinion that most of these discussions are merely the modern incarnation of the starving artist’s affection, and at best are merely the backroom gossip of a new publishing industry. This includes efforts by commentators and critics to engage us regarding the availability or frequency of comments, graphics, advertising, or any of the dozen other editorial policy issues that arise when blogging.

However, we make an exception and are willing to engage in a limited degree of navel gazing when it comes to evaluating the blogsphere, and the technology space it arises from, as a whole. It is a fundamental requirement of our mission mandate that we occasionally peer at the turtles upon which this Parallel World is carried.

One of the best and most authoritative sources by which we may observe the changes in this particular segment of communications space is Dave Sifry’s re-occuring State of the Blogosphere series. The respect accorded to this individuals writings is derived from his position as founder and CEO of Technorati – we always listen to any man that builds and owns a non-traditional collection platform.

A News Cycle Measured in Megahertz ....

.... We track about 1.2 Million posts each day, which means that there are about 50,000 posts each hour. At that rate, it is literally impossible to read everything that is relevant to an issue or subject, and a new challenge has presented itself - how to make sense out of this monstrous conversation, and how to find the most interesting and authoritative information out there.

Key highlights from his summary which are also of interest:

  • Technorati now tracks over 27.2 Million blogs
  • The blogosphere is doubling in size every 5 and a half months
  • It is now over 60 times bigger than it was 3 years ago
  • On average, a new weblog is created every second of every day
  • 13.7 million bloggers are still posting 3 months after their blogs are created
  • Spings (Spam Pings) can sometimes account for as much as 60% of the total daily pings Technorati receives
  • Sophisticated spam management tools eliminate the spings and find that about 9% of new blogs are spam or machine generated
  • Technorati tracks about 1.2 Million new blog posts each day, about 50,000 per hour
Needless to say, this is a daunting volume of material in which relevant and valuable signal is no doubt originating at this very moment. The challenge here goes beyond technologies a core questions of the community – stealing secrets or creating knowledge?

When one considers that this is just one fragment of the entire open source problem space, even the most veteran OSINT proponent must stand in awe at the task before him. One wishes Mr. Jardines’ new organization the best of luck in this endeavor.

For those with extra time on their commute and space on their iPod, a recent (January 2006) interview with Mr. Jardines is available via podcast.