Can I haz relevance?

The trouble with innovation is that while it solves a problem with the status quo in the hopes of making our world a better place, it typically creates a whole new set of issues that require new ideas to solve.

Take the invention of the automobile for example - clearly one of the milestone moments in human history, allowing people and things to travel greater distances in a reasonable amount of time. This simple invention however created the need to create a system to allow people to travel easily from one place to another (roads) and technology to manage and facilitate the flow of vehicles from one place to another (signals and traffic laws) and these innovations have lead to a whole other round of creative solutions - from dirt roads, to paved roads and then freeways for example. Not to mention the unforeseen consequences of these inventions that to this day are still challenges - traffic accidents, oil dependency and global warming for example.

This simple example is not limited to the creation of the automobile - think about the creation of the personal computer, which spawned the need to connect those devices (networking) and then the realization that we could bring those networks together so the worlds information and ideas could be shared in real time (the internet) which of course leads to the challenge of organizing that information (search engines, wikis and blogs). It is pretty amazing how far we have come from barely having computers connected to each other to spending the majority of our days using the Internet.

What of the unintended consequences then? I am not talking about the bad stuff on the Internet - you know like child porn, Nigerian investment schemes and (yea I went there). I am talking about the fundamental problem with having all of the world’s information at your fingertips – there is just too damn much of it! Tried searching for something lately? It’s a pretty big challenge to find what you are looking for most of the time. It seems like more and more you have to have a degree in Information Sciences to do a search online. There is just too many things out there that appear to be what I am looking for and I don’t feel like our current search engine options are really trying to help us make better searches.

Do a broad search, and you will be staggered by the number of results you are presented with. For example a search for “Mobile Phone Reviews” on Live Search produces 47,800,000 results. The same search on Google produces 41,900,000 pages.

Wow, over 40 million results? Really? What the heck am I going to do with 40 million results? Let me crack a beer here real quick and get started, I might be here for the rest of 10 life times reading all of these to make sure I get to the page I want to. Hold on a second, isn’t that what these search engines are supposed to do? Read through the depth and breadth of the internet, eliminating all the pyramid schemes and NAMBLA hotlists along the way, to present me with what I was looking for in the first place? Isn't that why we invented computers and the internet anyways, to do all the hard computations and analysis we can't do?

Now as someone who has seen the explosive growth of the internet in the last 15 years, it doesn’t surprise me that there are this many pages that are related to mobile phone reviews on the internet (I am sure it is actually far more, and that Google and Live are being somewhat selective). That is not my issue, it is not with how much information is out there - it is how it is being presented to us by the very search engines that are supposed to help us navigate the Internet. Clearly, it would appear that we have crossed the threshold where the current method of crawling and spewing results based on keywords, link backs and meta data isn’t working anymore – just like dirt roads were no longer an effective driving surface for cars. What I am really trying to say is - I think it is time for someone to pave the internet, but not in asphalt – let’s pave it in relevancy.

You see, I don’t know how to search and neither do you. It’s the simple truth. Sure, there are some people out there who can craft the perfect search phrase full of w/20’s and –this and +that’s. Maybe even I could sit here for a while and try 20 different keyword combinations and find what I was looking for, but why should I have to? Can’t we come up with a better way?

Could you image driving without street signs or traffic lights? That is how I feel it is to navigate the internet sometimes. The best I can do is slam on the gas and hope I don’t run into something I don’t want and somehow get to where I want to go. So I just keep making random turns, hoping to see that place I want to be. Sometimes I find it on the second turn, sometimes on the 10th. Every once and a while I get so frustrated by not being about to find where I wanted to go that I just turn the car around and head back home. Clearly this is not the most effective way to travel and it certainly is no way to find information.

To me, It seems that the first search engine that makes it easy for me to refine my search – helping me get closer to the best information on the topic I want, in a visual and efficient way, will take the Internet to its next logical evolution and maybe just maybe finally make it the effective storage space for the worlds information it was dreamt to be, as opposed to the impossible to navigate dumping ground the current search methodologies makes me feel like it is. Machine learning and reading context from the search terms I use is a start (several search engines and search prototypes are doing this now), but its not going to get us all the way there. At some point the computer can't do all of the work. I have to help a little bit (and I am willing to do that, if it is easy and results in a set of results that are more likely to answer my query. Perhaps someone will figure it all out  and then maybe someday, I can haz relevance. We will see...


Comments (3)
  1. It’s not about how many results are returned (which is really metadata to aide in seeing how specific your query is, and/or how popular your terms are), it’s about the first results page. To use your road analogy: imagine being on a road with 40,000,000 million intersections. A good search engine would make sure that the first ten intersections were most likely to be the correct turn.

    If you’re looking for hand-picked results, then perhaps you’d be more comfortable with a index rather than crawler-based search engine. is a good example. Instead of going from a large-to-small relationship, they start with no results on a given topic and only add the best results. This was the model that Yahoo! used in the early days, and metacrawler (then Google) took over because indices have obvious limitations.

    Search algorithms have come a long way in the last ten years. Clickthroughs from search/browsing histories factor in a lot more, as does white-hat SEO.

    I don’t understand the mobile phone reviews example — the results from both engines ARE relevancy ranked! There’s no magic bullet — if there were, it wouldn’t be called search. Trial and error will always be a source of frustration, but it’s in the best interest of information sources, search engines, AND end-users to mimimize dead ends.

    The improvement I imagine is tailor-made content based on what the search provider already knows you know, or what you don’t want.

  2. frances says:

    Brad –

    Hey buddy, glad to see you’re reading the blog 🙂 Thanks for the feedback!

    So I understand completely that today results are ranked by relevancy and on the website developers end SEO (Search Engine Optimization for the kids playing at home) can go a long way to making sure that there site is pinging the correct keywords and context to get high rank on the searches that are most related to the content on their site.

    My argument wasn’t really against the 40 million; it’s really against the first page. For those of us who are technically inclined, have a lot of experience and know how search engines function – we can put together a pretty good string of search terms to find what we are looking for, and we expect to have to click through a few results till we find what we want. This isn’t always the case, as I relatively frequently find myself unable to find what I was looking for in fringe cases, but in general we are pretty successful.

    The argument is more two sided, the first side for those who are less experienced and don’t really know how to search (the majority of the people on the internet). The second is the general lack of assistance I feel we get from search engines to help us get to where we are trying to go when we (the good searchers and the bad) are unable to find it.

    Some search engines provide a list of more refined keywords to the far right of the search box – the problem is these terms are pretty specific, and in my experience rarely if ever help to refine to search. I think there is a better way to do this, which has a better user experience, and helps to break a search into parts and drill down into specifics from there.

    I have also seen some of the search engines and technologies you talk about, but I haven’t been impressed with indexes (like Mahalo). I don’t have a fundamental problem with crawling, I just think search engines can do better in the User Experience after I type in my keywords and click "search" beyond presenting us with pages and pages of blue links on a white background.

  3. 話題の小向美奈子ストリップを隠し撮り!入念なボディチェックをすり抜けて超小型カメラで撮影した神動画がアップ中!期間限定配信の衝撃的映像を見逃すな

Comments are closed.

Skip to main content