Thursday February 16, 2006
The Guardian
If life without Google is hard to imagine for most web users, life without the blog search engine Technorati.com is probably unthinkable for inhabitants of the blogosphere. Where Google lets users find what they are looking for, Technorati enables bloggers to find out something even more precious: who has been linking to their posts.
Technorati is a classic case of scratching a personal itch. Back in 2002, David Sifry was a 33-year-old serial entrepreneur in Silicon Valley. After gaining a degree in computer science, a stint in Japan and another on Wall Street, he'd already launched a couple of companies and was busy setting up another. Along the way, he'd discovered blogs.
But, as Sifry says, blogging inevitably raises two questions: "What's everyone saying about me - what's the conversation going on around my stuff?" He tried to find out using traditional search engines: "They all failed miserably," he says. They failed because their approach - based around counting the number of links to a web page - "is fundamentally biased against things that are new, because it takes time for links to accumulate pointing to something." But in the blog world, it is precisely the new things - the most recent posts on other blogs - that are the most interesting, especially if you want to join in the conversation.
Traditional search engines send out a software "spider" to index all the pages on a website, whether or not they have changed. This is a huge task, so the time between visits of the spider is usually days - hopeless for the fast-moving world of blogs. A better method would be to index blogs as soon as they are updated, and only then. Sifry realised that a mechanism was already in place that would allow him to do precisely that.
'Ping'
"A huge amount of credit has to go to Dave Winer," Sifry says. Winer was one of the ur-bloggers and also the author of an early blogging program called Radio Userland. Winer had built a feature into Radio Userland that would prove key to Sifry's new kind of blog search. Winer "created something known as a ping," Sifry explains. "Whenever anybody who was using his tool, Radio Userland would update their blog, [the software] would automatically update a site called weblogs.com," owned by Winer. "It would send off a little notification message" - the ping - "saying 'hello, Bob's thoughts just got updated'." So all Sifry had to do was create a spider that went to weblogs.com, downloaded which blogs had been updated and then indexed those new blog entries. This formed the basis of what became Technorati (chosen because an earlier idea, "Digerati", was unavailable as a domain name).
Sifry then approached all the leading blog software vendors and said: "Look, if you guys build this [ping mechanism] into your tool, and have it ping Technorati [as well as weblogs.com] we'll go out and index your content literally within seconds of when the user posts." The software companies readily agreed. First, because "everybody who writes a blog wants to get found," as Sifry points out. Second, he says, because "I literally would write the code for people."
Rapid programming was crucial for the creation of Technorati itself, which began in November 2002 as Sifry's "weekend project" in his basement, alongside his latest startup, consulting to pay the bills and the arrival of his second child. Sifry's hacker past helped - one of his previous startups was Linuxcare, which had employed half the coding stars of the open-source world - but so did the ready availability of well-honed GNU/Linux software such as the Apache web server, the MySQL database and the PHP and Perl scripting languages, collectively known as the LAMP stack.
The constant time pressure Sifry was under had an important consequence. A few months after Technorati was launched, the 2003 invasion of Iraq took place. "It hit me like a lightning bolt," Sifry explains. "I don't have time to figure out what's the most interesting stuff to read, but there's like a million other bloggers, they know.
"What if I took the four thousand or so different mainstream news sources that are out there," he continues, "and then looked at all the people in the blogophere: what are the stories they are linking to the most in the last three hours?" The result was the first of what Sifry calls "discovery products", which appear on Technorati's home page.
"This is one of those things that I think is fundamentally different about Technorati," compared to Google or Yahoo, he says, since it is based on "understanding people and understanding time" - not just on static links between web pages.
Unique insight
Technorati's way of tracking blogs - 27m of them at last count - gives Sifry a unique insight into this world, which he reports on in his occasional State of the Blogosphere, posted on the Technorati blog. As well as problems like splogs (spam blogs) and spings (spam pings), one last week touched on the issue of blogfade: the fact that 50% of new bloggers have given up after three months. Sifry is not unduly worried. "The number that I find indicative of the true growth of the blogosphere is the number of daily posts. This is a much better way of being able to watch how much time people in aggregate are putting into writing on their blogs. Today, we're tracking about 1.2m posts a day." A year ago, it was a third of that.
Other notable trends include the growth of blogging in eastern Asia: last month, more posts were made in Japanese than in any other language - English-language posts represent about 28% of the blogosphere. In part, this is because Japanese bloggers tend to post lots of shorter entries, a habit that is spreading, Sifry says. "More people are using blogs as a sort of conversational medium, as opposed to the long-winded 'here's my 500 to 1,000-word essay' medium." As a result, the average number of links in each post is dropping.
Another development concerns who is using Technorati, alongside all those bloggers checking their links and the time-challenged looking for news. "We have a lot of companies that use this service who want to be able to track 'what are people saying about my company? What are they saying about my products? Hell, what are they saying about my competitor's products?'" Other regular visitors include the "cool-hunters", desperate for the next trend in music or fashion, and journalists seeking stories and sources.
Technorati makes its money from sponsorship and advertising on the main site, as well as syndication revenues from titles such as the Washington Post, Newsweek and Der Spiegel: alongside a news story, Technorati provides real-time links to who is blogging about it.
Sifry's company is still privately held: "We did a little bit of an angel round with some smart Silicon Valley investors in 2003," he explains, and in 2004 he got funding from two venture capital firms.
Sifry is evasive about any possible stock market float, and laughs uproariously at the idea of being made an offer he can't refuse by somebody like Google. But given the rise of blogging as the new face of the web, and Technorati's central position in the blogosphere, going public or being acquired is surely just a matter of time.
Curriculum Vitae
Age 37
Education Graduated in 1991 with a degree in computer science from Johns Hopkins University, Baltimore, Maryland.
Career
1991-94 Mitsubishi Electric, Japan
1994-96 Advanced Portfolio Technologies, Wall Street, New York
Then, as an independent consultant, Sifry founded:
1996 Securemote
1998 Linuxcare
2002 Sputnik
2003 Technorati
Hobbies Amateur photography. Examples of Sifry's work can be found at www.flickr.com/photos/dsifry
Glyn Moody blogs at opendotdotdot.blogspot.com
No comments:
Post a Comment