The May 2006 issues of Business 2.0 has a few interesting stories that while on the surface don’t seem to be related are in fact incredibly related. However, first, there is good news for all those DBAs out there. One of the articles in this edition is called “The Next Job Boom”. The article contains a top 10 list of hot jobs. Five out of the top 6 are technical jobs – I guess out-sourcing isn’t impacting technical jobs as much as people worried it would. Four of the top 10 are medical jobs and number 10 is college professors. What’s really interesting is the sixth spot is DBAs, with an expected job growth of 38.2% through 2014. The data for this list comes from the Bureau of Labor Statistics and is specific to college-educated workers.
There are a couple of interesting assumptions that had to go into this figure. The first assumption is that data will continue to grow at an incredible rate with most of it being stored in databases. The equation is simple: more data = more databases = more DBAs. I absolutely agree that data will continue to grow at an incredible rate and that data must be stored in a relational engine. The second assumption is that database growth will out pace our (Microsoft, Oracle and the multitude of other software companies that build database tools) ability to make databases easier to manage thus allowing DBAs to scale. This assumption is pretty much crap. DBA tools today are very primitive and there is a lot of room for improvement. I predict between now and 2014 you’ll see vast improvements in these tools.
On to some really interesting stuff. The articles that I found interesting and related are “The Untapped Search Business”, “The Innovators: Lighting Up Microsoft”, and “The eBay of Swap”. The article on search, by Paul Kedrosky, is what really started my thinking. Search is a hot topic these days and there are numerous start-ups trying to come up with the magic elixir. Essentially the article says that Microsoft and Google are wasting money and resources with their attempts to improve relevance. It makes a couple of points about the search interface – almost arguing with itself – the UI is not so interesting; the UI is very interesting. And finally comes to the conclusion that the interesting work in search is being done by companies that are launching search engines for specific topics like health care and travel. At first I thought this was interesting. I went to Kosmix.com and looked at their product. In short, I wasn’t impressed. I don’t doubt they have some really smart people working there and there is some nice technology under the covers, but it was a big yawn and I don’t think I’ll go back there. But you should take a look for yourself. The “product” is still in Alpha (whatever that means) so there is presumably time to rescue it.
The second article, “The Innovators: Lighting Up Microsoft”, is an interview with Ray Ozzie. I should come clean and state that I’ve never been a fan of Lotus Notes. Notes is a very interesting concept with a very poor implementation. But that’s off topic. I have incredible respect for Ray and I’m proud to have him at Microsoft. MS is a better company with Ray than without. The most interesting point that Ray makes in the article is this: to build great software you need to start with the user experience – what are people attempting to accomplish – and aim for the minimum necessary to accomplish that experience. He points to the iPod as the prototypical example of this and calls the approach scenario-based design.
I see it go on all day long at Microsoft, not scenario-based design, but rather the opposite. We dive very deep in to the technology focusing on the design of the internals (how should this API interface look) rather than focusing on how people are going to use it. This judgment may tick off some people and I’m sure they’ll defend their approach, but they’re wrong. They don’t do scenario-based design and they’re more interested in the underlying architecture than how people will use it. It’s not that the architecture isn’t important, it is. It’s that the architecture is the means to the end not the end. If you don’t know where you’re going who cares how you’re getting there.
The final article is “The eBay of Swap”. The article is about a group of companies building high-tech bartering systems that let consumers get the goods they want without paying a dime. This model is very interesting and has a lot of merit. But the point I want to make has nothing really to do with the business model but rather with the fact that under this model is a relational database. I’m sure these companies didn’t start by saying, “hey, we have this relational DB, what kind of company can we build from it?” It’s almost laughable to say that aloud. Rather, they said, there are lots of people who want something but rather than exchanging money they have other goods to exchange. Hmm, that sounds an awful lot like a scenario. The relational database became a means to the end not the end. I haven’t tried out any of these sites so I can’t speak to how easy or difficult they are to use. But the scenario is compelling.
So here’s the punch line. The GUI for search is the most important part of the search. It makes the product. Just like the GUI sitting on top of the relational engine is more important (I’m sure someone in SQL Server land is going to give me a hard time for this statement) than the relational engine. Yes, the DB plays a critical role in the application but I have never heard anyone say “this application is cool because it uses SQL Server.” Rather the application is cool because it solves a real need in a very elegant way. There is a real need for search. But do I need 68,879,976 results to my search sorted in some unknown fashion? Absolutely not. What I need is those results organized in a way that matches my scenario. Kosmix and others are trying to do just this but they’re not there yet. The results are categorized, but they’re still presented in a Google-like way. Which isn’t very helpful. Live.com and Google are still way out in the weeds.
I don’t know what the answer is to the search problem but I do know that relevance is the wrong way to define the scenario. Local versus non-Local is also not the right-way to define the scenario. I don’t know what’s going on within the search team at Microsoft, but I know there are a lot of really smart people working on it. I just hope they’re not doing technology for the sake of technology but rather technology to support my scenario. Relational databases are cool and allow us to do amazing things, but they aren’t the application – they play a supporting role. The search engine also isn’t the application but rather a supporting role. We need to internalize this and change the way we approach solving problems. Once we do that awesome things will happen.