Is Microsoft a search company now?

I recently chastised Microsoft for suddenly pulling the plug on two of its web projects, but now I'm starting to see the method in Microsoft's madness. Don't get me wrong -- Microsoft still deserves to be chastised. It's just that I think I have a better idea of what the company is planning regarding its web business. Microsoft seems to be realigning its web strategy...towards search! This move definitely bucks traditional wisdom according to which the search market is already pretty well locked up by Google. Evidently Microsoft sees an opportunity that others have missed.

Still skeptical? To me, this realignment towards search is the only way I can explain Microsoft's recent moves. Let's begin with last year's launch of the Live Search Club, the site that rewards people for playing games that just happen to force searches. This was a roundabout way of getting more searches conducted and more people using Live Search. This project is still going strong -- I regret that I missed the opportunity to write up the double tickets promotion that took place yesterday. I participated even though I'm still unhappy about the Live Search Books and Live Expo closings. On to 2008. The biggest Microsoft story of the year of course has been its attempted acquisition of Yahoo which just happens to operate the Internet's second most popular search engine. Soon after that failed, Microsoft started talking to Yahoo about another possible deal, which might include an acquisition of Yahoo Search and other selected properties but not the rest of Yahoo. I'm sure Microsoft had more in mind than just increasing its search share when it started bidding for Yahoo, but had an acquisition of Yahoo's search business happened it would have left Microsoft in control of the second and third most popular search engines. If Microsoft had just wanted eyeballs, it probably could have acquired AOL more easily and more cheaply; search was definitely a motivating factor in all this. In May, Microsoft announced its Live Search cashback program which allows users to get some money back on their purchases. Similar to the Live Search Club, Live Search CashBack gives people incentives to search, but like any rebate program it only works when purchases are made. Why just reward people for searching when you can reward them for doing what you really want them to do? Advertisers should love a search engine whose users like to buy stuff. This is not a surefire success by any means -- cashback might spoil people to the point that they will come to expect to get money back off every online purchase (and it does cost Microsoft potential revenue if nothing else), and the whole thing could end up just attracting bargain shoppers instead of the wide base of people Microsoft is probably hoping to draw to its search engine. Perhaps the real genius of this move lies in the timing: rough economic times have turned many former shopaholics into bargain shoppers so any and all cashback programs will be welcomed by many. At any rate, cashback should increase Microsoft's search share and encourage a lot more spending...good for the economy, good for Microsoft. If really successful, cashback could have a disruptive effect on search advertising and perhaps force other search engines to offer similar programs. I don't expect to see that happen; rebates have been around for an awful long time, after all. It's not like Microsoft is trying something people never dreamed was possible, but the concept of combining rebates with search is a little bit new. Most recently, Microsoft made a deal with HP which will put a Live Search toolbar on new PCs starting next year. Nothing really special about that -- all the search engines make deals with PC manufacturers. It shows Microsoft is serious about search, though, and every such move is going to increase its search share.

All this leads me to believe that we haven't seen the last of the search wars. Still, I still don't necessarily think it is wise for Microsoft to scale back its other web projects just to focus on search. How many people use Google Docs more than they use Google's search engine? I would bet a fair number do now. How many people use Flickr or Delicious more than Yahoo's search engine? Lots. In fact, plenty of people who use those services don't search on Yahoo at all. Microsoft will be at a disadvantage if its competitors have hundreds of destinations that each attract users while it places all of its eggs in just a few baskets. The closings of Live Expo and Live Search Books just give users reasons to go elsewhere. Still, I predict Microsoft's search engine share will increase in the coming months. I could see them overtaking Yahoo, perhaps by next year, if they continue to be creative and aggressive. At the moment, though, I think the search share rankings accurately reflect the qualify of each search engine: Google is better than Yahoo which is better than Live Search. Live Search is decent now, but there's still too much crap that rises to the top there and too much of the Web it doesn't index. All it takes is one search that doesn't guide you to what you're looking for to cause you to think about changing search engines. As long as Google does search better, it will be hard for people to abandon it no matter how many carrots the other search engines dangle.

AOL wants your two cents at the Opinion Place.

I imagine that Google or Yahoo could become killer market research companies if they wanted. They certainly both have large and diverse audiences which could meet most any demographic need. For now, unfortunately, you're out of luck if you want to be paid for giving your opinion to Google and Yahoo about anything other than Google or Yahoo. I suppose I can understand why it might not be that appealing for a company to do market research by working with another big company, possibly a competitor. Microsoft wouldn't want to call up Google Surveys in order to research how people are using Office, for instance. That's likely the reason why so many small market research companies are thriving by working with some very big companies from all kinds of industries.

For some reason, AOL is the exception to the rule I just suggested. They're a big company, but they still do market research in partnership with DMS Research. AOL's survey site Opinion Place is one of the best sites of its kind. The way it works is simplicity itself. First, you go to the site and create an account. You'll be asked fill in a fair bit of personal information as that is the only way you can be hooked up with surveys that target specific demographics. You'll also be asked how you wish to be compensated for your participation -- I like the payments by PayPal myself, but AOL subscribers and frequent fliers might prefer credits or miles to cold, hard e-cash. Then you can take a survey. It's a very straightforward process; sometimes you're matched with a survey, and sometimes you're not. After all is said and done, Opinion Place will let you know when you can take another survey.

Taking surveys probably won't pay your Internet bill, but it is a fun way to burn some time and earn a little extra cash at the same time. It's actually useful work you're doing as well, as your answers will, in aggregate with the answers of many others at least, influence the products and services companies provide. You also sometimes get to find out about cool "coming soon" stuff through surveys; sadly, you are generally strictly forbidden from discussing anything top secret. To give a non-specific example, I once got to test out a beta version of what has become one of my favorite multimedia sites as part of my participation in a survey. Surveys are pretty cool!

I've often felt that the big Internet companies are so keen to make money off the little guy that they forget that they can sometimes profit right along with the little guy if they're willing to work with him. Amazon understands this; that's why they have their affiliate program, Mechanical Turk, and the Honor System. Google gets it; that's why AdSense is open to everyone. Making money with most other big Internet companies isn't quite so easy if you don't want to apply for a job. AOL has successfully transitioned from an Internet service provider to a web content provider, but their business model is still largely based on profiting from an audience that consumes rather than produces. This being the case, I must commend AOL for showing a willingness to do business a little differently when it comes to the Opinion Place. Thanks, AOL, for providing a great opportunity for people to share their opinions and earn a few extra dollars in the process.

Microsoft is in retreat mode.

Not too long ago Microsoft was on the verge of becoming a much bigger Internet company. It was about to acquire Yahoo, about to finally give Google some real competition...but the Yahoo deal never happened and Microsoft's web presence has been shrinking or at least narrowing ever since. I think it is likely that Microsoft is still intent on acquiring Yahoo somewhere down the line, but for now they seem to be too busy decimating their own web properties to worry about decimating Yahoo's. Maybe they're just streamlining or refocusing, but shutting down two major projects in two months is not what I expect to see from a company that allegedly wants to dominate the Internet.

The first victim was Live Search Books. I remember thinking what a gutsy move this launch was back in 2006 because Google Book Search already existed and already rocked. Microsoft was essentially saying they could go toe-to-toe with Google and perhaps even do book digitization and search better than Google. The abrupt closing of the site in May made it clear that Microsoft couldn't be competitive in this space. What puzzles me is that surely Microsoft couldn't have believed that such a project would ever be a lucrative moneymaker. They had to know going in that this needed to be about providing a useful service, creating good will, and showing their ability to compete with Google. At what point did Microsoft decide, "Well, it doesn't matter if we no longer provide a useful service, destroy the good will we've built hitherto, and show that we can't compete with Google." It's just strange to me...very strange.

June's abandoned child was Windows Live Expo, Microsoft's foray into the online classifieds business. Classifieds seems to be a tough business to break into -- even Yahoo's site wasn't able to survive. Still, someone has to challenge Craigslist sooner or later. Microsoft evidently decided it wasn't up to the task. Personally, I thought Live Expo was appealing visually and easy to use. Its weakness and probably the reason it was shut down is that it never became a huge site. Not being a huge site is a real disadvantage in the classifieds space; people want to be able to browse local listings no matter where they are, and that just isn't possible if people from all over aren't selling stuff. Perhaps the fact that Craigslist isn't a direct Microsoft competitor in other areas, unlike Google, influenced this move, though that type of thinking didn't save Live Search Books. Both sites probably had difficulty generating revenue, but Microsoft should have known from the start that neither site was likely to be profitable from the beginning...especially not a freaking book search site! At the end of the day, closing any site is a sign of weakness and a message to your user base saying, "Don't trust us!" Microsoft may have big pockets, but it needs to develop more staying power if its online projects are ever to reach their true potentials.

Who will search the microblogs?

I noticed something interesting when I was searching for Jaiku related blog posts with Google Blog Search recently. Many of the results of my search weren't exactly what I was looking for -- I wanted to read long-form blog posts related to Jaiku -- but they were nonetheless quite relevant to my search term. Instead of just giving me blog posts about Jaiku, Google also gave me returned actual microblog posts on Jaiku. To me this is interesting because Google seems to be treating microblogs and blogs as similar entities that can both be searched on Google Blog Search. Personally, I tend to think that microblogs and blogs are quite different species and that search engines should treat them accordingly.

Why shouldn't microblogs and blogs be lumped together? I'm looking at this issue primarily from a searching perspective. When I do a blog search about something, I expect to find more or less developed articles and/or collections of links. I'm not going to be satisfied with a sentence or two that happen to contain my keyword(s). If I'm looking for long-form content, microblogs appearing in my blog search results are simply noise. Even if I did want microblogs to be in the mix, filtering the useful microblog postings from the chaff is an unusually difficult challenge. Useful blog posts will attract links on the outside Web; useful Twitter postings probably won't, though they might be responded to more. How does a search engine compare blog posts having inlinks with Twitter tweets that aren't linked to? Does one give more weight to Twitter folks who have more followers or more links to their Twitter feeds than others? While intelligent and useful searching of microblogs is important, I don't think the solution involves treating conventional blogs and microblogs as if they were the same. Instead, I think we need "conversational search" that is just for microblogs, forums, and any other searchable forms of online chat. Thus far, the giants have been slow to recognize this need.

You might well ask yourself, "Is this really necessary?" After all, don't search engines search everything...isn't that what they're supposed to do? Sure. When I go to a search engine, I do expect to see everything in the general web index. Specialized search -- be it image, video, blog, or whatever -- makes things easier for me when I really want to narrow things down, though. If I do a general web search for tennis, then I expect to get a bunch of different stuff back: tennis news and results, the rules of the game, shops selling tennis supplies, etc are all appropriate first page search results for my very broad query. If I do a blog search for tennis, then I expect to get back more opinionated but still well-developed content. I don't expect just the news and results, but rather different personal takes on the news and results. I don't expect to find stores, but rather opinions about the stores and general posts with affiliate links. If I do a microblog search, I'm looking for small morsels of content: "Tennis sucks," "Tennis rocks," and "Tennis is hard on the knees." A tweet might convince me to start following someone and make a new friend. Alternatively, maybe I'm searching the microblogs just so I can explore a kaleidoscope of thought. Are people liking tennis more or less these days? Microblog search can give us a more personalized picture of shifting opinions than Google Trends can. No search engine can read minds, but I think it's safe to say that someone who is looking for blog posts about tennis does not want his search to lead him to a "Tennis sucks" microblog post. That post could be just what someone else is looking for, but I think more often than not microblog posts will just be adding noise to blog search engine results. This isn't a problem if we have conversational search.

There are already some quite decent Twitter search engines out there. At least one of them, Summize, unabashedly says that conversational search is what it does. The problem I'm seeing with these engines is that they're only searching Twitter right now. Twitter is the top dog in the microblogging world, for sure, but that doesn't mean other conversations should be ignored. As I mentioned earlier, I even think forum posts should be a part of conversational search (after all, they ARE conversations!). They already quite often show up in general web search results and have often helped me solve very specific problems; frankly, forums have proved a lot more useful to me personally over the years than microblogs have so far. There are other conversations going on elsewhere on the Net that could be indexed: for instance, I think IRSeek, which searches IRC chats, is a great service though it's been somewhat controversial. A really good conversational search engine will look for conversation everywhere and index it like mad.

Of course, the very phrase "conversation search" implies that microblogging is all about conversations. Truthfully, it isn't always. You can certainly tweet about anything you want without having any followers. You can also Jaiku haikus to your heart's content -- in that case, you're microblogging to express yourself, not to conversate with others. In such instances, perhaps those particular microblog posts would be more at home amongst traditional blog posts rather than forum posts and IRC logs. Perhaps, then, "conversational search" isn't the answer, but I still think we need a way to conveniently search microblog posts and that it is best to segregate regular blog posts from microblog posts. Whoever does it will have to tackle some tough questions. I already mentioned the difficulty in determining how to rank microblog posts. What about the difficulty in actually determining what a microblog is? I assume this determination would be based on platform (for example, Wordpress = blog while Twitter = microblog), but if someone writes really short posts on a Blogger or Wordpress blog isn't that person really microblogging instead of blogging? Anyway, it'll be very interesting to see if one of the big Internet companies will tackle this problem or if one of the independent search engines will dominate this still fairly fringe interest instead. Google seems to be the most natural home for conversational search to me, especially since it has its own microblogging service which needs to be promoted more, but it would be a good addition to Yahoo! Search or Live Search as well.

Twitter may be down frequently, but Jaiku is always in beta.

Twitter has quickly become the king of the microblogging world. The darling of early adopters, Twitter is increasingly attracting mainstream Internet users as well. It should keep growing, too, because people are going to want to use the same service as their friends are already using. Twitter does have a weakness, though, and it's not something it can hide: it goes down, and fairly regularly at that. For the most part, Twitter users have proven to be an understanding lot; many of them realize that Twitter is an independent venture that has grown very big very quickly. Still, there's little that's more annoying than an unreliable communications network; Twitter's downtime ought to be fueling competition in the microblogging world.

Google's microblogging acquisition, Jaiku, has not taken advantage of Twitter's weakness. Although still frequently mentioned as being Twitter's primary competitor, Jaiku hardly offers a refuge for those seeking a more reliable microblogging alternative. If you want a Jaiku account, you have to go to the website, request an invitation, and wait. (You could also get invited directly by an existing Jaiku user, but invitations are limited.) If you want a Twitter account, you go to the website, signup, and start tweeting. By the time Jaiku sends you an invitation, you could already have built up a network of Twitter friends. Granted, Jaiku is in beta, still a work in progress. Google has missed the boat by not putting more resources into its microblogging platform; there may never again be as good a time to build up such a service. In all likelihood, Twitter will overcome its uptime issues and consolidate its position as the top microblogging service. Jaiku may have to settle for second place or worse if it doesn't come out of beta soon or at least start allowing open signups.

On the other hand, it is indisputable that Google achieved a lot of success following a similar strategy with Gmail. The invitation-only model there created massive amounts of interest prior to the service opening up. There isn't such a huge drive for Jaiku invitations as far as I can see, and I think this is largely due to the perception that Jaiku is not that different from Twitter. Gmail quickly gained a reputation as being something revolutionary; Jaiku, on the other hand, seems to be widely considered merely a pretty Twitter alternative. Additionally, I think many people are going to prefer to stick to one microblogging service; in contrast, few people seem to have just one email address nowadays. I could be wrong, but I think Jaiku would be better off if it were open and out of beta.

Yahoo! tracks backlinks where Google fears to tread.

If you've ever created a web site, you've also probably gone to the search engines to check to see who, if anyone, is linking to you. If you use both Yahoo! and Google, you've probably noticed a big difference in number of links to your site that these two engines are reporting. Although Google is the world's most popular search engine, Yahoo! is much more thorough when it comes to counting backlinks. Indeed, many people who use Google as their main search engine use Yahoo! only to look for backlinks -- it's great to be the best at something, isn't it?

I'm not so sure Google even wants to be the best backlink counter on the Net, however. The "no-follow" attribute that is added to more and more links these days is part of the problem. I totally understand why Google doesn't want paid links and spammy links to improve a site's ranking in the search results, but I don't like the idea of the Google bot seeing a link attribute and saying to itself, "Well, I won't look there." The Google bot is supposed to look everywhere. It should know about every link on the Web, in my opinion, whether those links be no-follow or do-follow. If people want to keep their content off the search engines, robots.txt and password-protecting pages are methods that still work. Granted, no-follow makes the process of avoiding the Google bot easier -- it even has the effect of democratizing the process because users of Blogger and Geocities and other similar services as well as non-technically inclined web publishers everywhere can easily utilize no-follow links. Still, the Web is basically a public place, and I'm just not convinced at all that that many people want to have their content on the Web freely accessible to all but still hidden from search engines. After all, people who want to share content within a group but not with the outside world can use services like Google Docs and Blogger to do just that and totally control who can access their content. In my opinion, no-follow shouldn't be taken too literally; the search bots should still follow, but they should only consider do-follow links to be "votes" for a given website that need to be reflected in the search result rankings. As no-follow begins to be used more and more by people who simply don't want to pass PageRank around (except, perhaps, to their own sites and to their friends' sites), I think it'll become only more important that search engines know where all links on the Web lead.

Yahoo's more open-minded attitude towards no-follow lets webmasters and other interested parties find links, no-follow or not, that Google doesn't seem to even know exist. It's really not just about no-follow; Yahoo simply takes tracking and reporting links in general more seriously than Google does. When I go to Google and type in "link:del.icio.us" I want to find out who is linking to the world's biggest social bookmarking site. Google does find more than 400,000 links, so that's plenty to keep me busy and an indicator of just how popular Delicious has become. When I go to Yahoo! Site Explorer to explore del.icio.us, however, I find over 33 million links which is on another level. The difference in reported links is staggering for all sites, large and small. I still like Google for search better than Yahoo overall, but when it comes to counting links Yahoo! has a clear edge. Eventually, that link advantage could help Yahoo improve its search as well.