I am currently in Neuss at an HP event. HP is a sponsor of our MTC and the event is all about the HP EVA program. That means: Storage, more storage, terra bytes, peta bytes...lots.
I was talking to one of the HP guys who is involved in the development of the product line. He was talking about the fact that typical web 2.0 companies come up with the need of low cost storage which is able to grow in unknown heights. Think about (just to name the extremes) Flickr or MySpace?? They started quite small and grew within months...
Then I thought about the technical requirements for a platform in this world. Overall.
I identified two distinct topics:
Diversity and Scale Out.
Diversity in the sense that it is not sufficient to have HTML and HTTP only. It starts with streaming media which needs more than HTTP only 😉 You have to support a variety of clients and identification models. AJAX adds web services (in one or the other way (SOAP or REST, you decide). RIA finally ends in Silverlight or Flash. Looking on the server: Backend in J2EE or .net, frontend in PHP, ASP.NET or Ruby on Rails?? Databases?? But please easy to setup and robust and able to...
Scale out in the sense that an application once written can grow up in unknown heights. What I learn at HP was that a massive growth in data has to be expected, too. So you need to be able to add processor power and storage. It was interesting to hear that the frontiers of computer science a few months ago today are within the reach of every mid-sized web company. We all know that the frontiers of hardware a subject to change every few weeks. But here we hit some harder barriers. Massive scale out is subject of intense research for years or even decades now.
Even more interesting is that most of the technologies used by modern startups today are designed without scale out in mind. Certainly they have diversity in mind. So it is no wonder that I regularly hear that they found single points of failure or timing problems within their apps.
In one point (not the only but the most remarkable) we - the HP storage professional and I - agreed: This industry gives us both enough space and fields to discover for our whole live 😉