Hitting the search relevance wall

There is a curious fenomenon going on in the search arena today - something I call the "hitting the search relevance wall". The thing is that most search companies are getting so much better that it is hard to distinguish between them in terms of quality. The days when Google had the best search relevance are over.

It's no wonder why. There are only a few links that you can display in the first page of your search results. And, no matter how good your relevance technology is, you will have to display the same links over and over, no matter what search engine you use. Therefore, if all search engines are getting to the "minimum bar" of displaying correctly the first links, then it is almost impossible to distinguish between them, if all you need is a few good links in the first page. And isn't this the most frequent way we are using these search engines? It's like hitting a "wall" when spending money on more and more research in improving search relevance - at some point you get less and less benefit no matter how much money you pour out in the technology, because the first links should be always almost the same.

The latest survey results from Keynote Systems are confirming this fact (the emphasis in bold is mine):

However, Keynote said it found that actual search results returned by the five search engines do not differ significantly. Actual consumer success in doing complex searches showed that the performance of Lycos, Ask Jeeves and Microsoft's MSN was as good as Google and Yahoo.

The article also mentions that MSN Search is getting visibly better in the last few months.

So, if search relevance won't be anymore a differentiator between search engines, then what's left? Plenty of cool things. Local search is one - and here I personally think that Virtual Earth rocks. Of course I am a MS employee, but, hey, am I allowed to love this site :-)? (Of course, Google Maps rocks too, and they have all my sympathy for inventing the draggable maps in DHTML). Also, new features like the integration between White/Yellow pages and local search are cool too. MSN Spaces is also something that spreads like wildfilre. Flickr is making inroads too.

There is plenty of room for innovation, and right now, the competition between Google, Yahoo, MSN and others is higher than ever. Good times to live in...