Who will search the microblogs?

I noticed something interesting when I was searching for Jaiku related blog posts with Google Blog Search recently. Many of the results of my search weren't exactly what I was looking for -- I wanted to read long-form blog posts related to Jaiku -- but they were nonetheless quite relevant to my search term. Instead of just giving me blog posts about Jaiku, Google also gave me returned actual microblog posts on Jaiku. To me this is interesting because Google seems to be treating microblogs and blogs as similar entities that can both be searched on Google Blog Search. Personally, I tend to think that microblogs and blogs are quite different species and that search engines should treat them accordingly.

Why shouldn't microblogs and blogs be lumped together? I'm looking at this issue primarily from a searching perspective. When I do a blog search about something, I expect to find more or less developed articles and/or collections of links. I'm not going to be satisfied with a sentence or two that happen to contain my keyword(s). If I'm looking for long-form content, microblogs appearing in my blog search results are simply noise. Even if I did want microblogs to be in the mix, filtering the useful microblog postings from the chaff is an unusually difficult challenge. Useful blog posts will attract links on the outside Web; useful Twitter postings probably won't, though they might be responded to more. How does a search engine compare blog posts having inlinks with Twitter tweets that aren't linked to? Does one give more weight to Twitter folks who have more followers or more links to their Twitter feeds than others? While intelligent and useful searching of microblogs is important, I don't think the solution involves treating conventional blogs and microblogs as if they were the same. Instead, I think we need "conversational search" that is just for microblogs, forums, and any other searchable forms of online chat. Thus far, the giants have been slow to recognize this need.

You might well ask yourself, "Is this really necessary?" After all, don't search engines search everything...isn't that what they're supposed to do? Sure. When I go to a search engine, I do expect to see everything in the general web index. Specialized search -- be it image, video, blog, or whatever -- makes things easier for me when I really want to narrow things down, though. If I do a general web search for tennis, then I expect to get a bunch of different stuff back: tennis news and results, the rules of the game, shops selling tennis supplies, etc are all appropriate first page search results for my very broad query. If I do a blog search for tennis, then I expect to get back more opinionated but still well-developed content. I don't expect just the news and results, but rather different personal takes on the news and results. I don't expect to find stores, but rather opinions about the stores and general posts with affiliate links. If I do a microblog search, I'm looking for small morsels of content: "Tennis sucks," "Tennis rocks," and "Tennis is hard on the knees." A tweet might convince me to start following someone and make a new friend. Alternatively, maybe I'm searching the microblogs just so I can explore a kaleidoscope of thought. Are people liking tennis more or less these days? Microblog search can give us a more personalized picture of shifting opinions than Google Trends can. No search engine can read minds, but I think it's safe to say that someone who is looking for blog posts about tennis does not want his search to lead him to a "Tennis sucks" microblog post. That post could be just what someone else is looking for, but I think more often than not microblog posts will just be adding noise to blog search engine results. This isn't a problem if we have conversational search.

There are already some quite decent Twitter search engines out there. At least one of them, Summize, unabashedly says that conversational search is what it does. The problem I'm seeing with these engines is that they're only searching Twitter right now. Twitter is the top dog in the microblogging world, for sure, but that doesn't mean other conversations should be ignored. As I mentioned earlier, I even think forum posts should be a part of conversational search (after all, they ARE conversations!). They already quite often show up in general web search results and have often helped me solve very specific problems; frankly, forums have proved a lot more useful to me personally over the years than microblogs have so far. There are other conversations going on elsewhere on the Net that could be indexed: for instance, I think IRSeek, which searches IRC chats, is a great service though it's been somewhat controversial. A really good conversational search engine will look for conversation everywhere and index it like mad.

Of course, the very phrase "conversation search" implies that microblogging is all about conversations. Truthfully, it isn't always. You can certainly tweet about anything you want without having any followers. You can also Jaiku haikus to your heart's content -- in that case, you're microblogging to express yourself, not to conversate with others. In such instances, perhaps those particular microblog posts would be more at home amongst traditional blog posts rather than forum posts and IRC logs. Perhaps, then, "conversational search" isn't the answer, but I still think we need a way to conveniently search microblog posts and that it is best to segregate regular blog posts from microblog posts. Whoever does it will have to tackle some tough questions. I already mentioned the difficulty in determining how to rank microblog posts. What about the difficulty in actually determining what a microblog is? I assume this determination would be based on platform (for example, Wordpress = blog while Twitter = microblog), but if someone writes really short posts on a Blogger or Wordpress blog isn't that person really microblogging instead of blogging? Anyway, it'll be very interesting to see if one of the big Internet companies will tackle this problem or if one of the independent search engines will dominate this still fairly fringe interest instead. Google seems to be the most natural home for conversational search to me, especially since it has its own microblogging service which needs to be promoted more, but it would be a good addition to Yahoo! Search or Live Search as well.