12 May 2008

Yahoo! tracks backlinks where Google fears to tread.

If you've ever created a web site, you've also probably gone to the search engines to check to see who, if anyone, is linking to you. If you use both Yahoo! and Google, you've probably noticed a big difference in number of links to your site that these two engines are reporting. Although Google is the world's most popular search engine, Yahoo! is much more thorough when it comes to counting backlinks. Indeed, many people who use Google as their main search engine use Yahoo! only to look for backlinks -- it's great to be the best at something, isn't it?

I'm not so sure Google even wants to be the best backlink counter on the Net, however. The "no-follow" attribute that is added to more and more links these days is part of the problem. I totally understand why Google doesn't want paid links and spammy links to improve a site's ranking in the search results, but I don't like the idea of the Google bot seeing a link attribute and saying to itself, "Well, I won't look there." The Google bot is supposed to look everywhere. It should know about every link on the Web, in my opinion, whether those links be no-follow or do-follow. If people want to keep their content off the search engines, robots.txt and password-protecting pages are methods that still work. Granted, no-follow makes the process of avoiding the Google bot easier -- it even has the effect of democratizing the process because users of Blogger and Geocities and other similar services as well as non-technically inclined web publishers everywhere can easily utilize no-follow links. Still, the Web is basically a public place, and I'm just not convinced at all that that many people want to have their content on the Web freely accessible to all but still hidden from search engines. After all, people who want to share content within a group but not with the outside world can use services like Google Docs and Blogger to do just that and totally control who can access their content. In my opinion, no-follow shouldn't be taken too literally; the search bots should still follow, but they should only consider do-follow links to be "votes" for a given website that need to be reflected in the search result rankings. As no-follow begins to be used more and more by people who simply don't want to pass PageRank around (except, perhaps, to their own sites and to their friends' sites), I think it'll become only more important that search engines know where all links on the Web lead.

Yahoo's more open-minded attitude towards no-follow lets webmasters and other interested parties find links, no-follow or not, that Google doesn't seem to even know exist. It's really not just about no-follow; Yahoo simply takes tracking and reporting links in general more seriously than Google does. When I go to Google and type in "link:del.icio.us" I want to find out who is linking to the world's biggest social bookmarking site. Google does find more than 400,000 links, so that's plenty to keep me busy and an indicator of just how popular Delicious has become. When I go to Yahoo! Site Explorer to explore del.icio.us, however, I find over 33 million links which is on another level. The difference in reported links is staggering for all sites, large and small. I still like Google for search better than Yahoo overall, but when it comes to counting links Yahoo! has a clear edge. Eventually, that link advantage could help Yahoo improve its search as well.

No comments: