Developers are not clueless…….

Ok, so I’ve now seen a bunch of posts on the subject of “why developers don’t build scalable systems”. Sure, it’s been phrased different ways:

“Why don’t .NET developers grok scalable distributed computing?
Disinterested Programmers
The Middle 70%

Ok, enough links and enough beating on this poor dead horse. I’d like to point out that I do believe that distributed computing technologies are important, just not to everyone. I also would agree that there are developers out there who should probably have a better understanding of distributed computing technologies and don’t (this doesn’t mean that everyone doesn’t).

I worked as a consultant in a past life and I was had the opportunity to see, add to, repair, and complain about systems that were built by someone who did not know how to architect a scalable application. That said, I think that you are a) preaching to the choir as most people who read blogs are pretty “up on things“ and b) referring to a small percentage of applications (and developers by extension) that actually need to scale in any major way.

Virtually all of the non-scalable applications that I saw (your mileage may vary) in the wild were not scalable because of poor database design and/or inefficient coding routines. The issues that prevented these applications had very little to do with the fact that they did not utilize a properly designed distributed architecture. Sure, some of them could have benefitted from a distributed architecture but that was not the overriding problem. We could make the custom software world a better place if we:

1) Keep all “developers“ out of the database. Let someone who knows how to design a database do most of the design work. Developers, you folks should be providing input in the process and pointing out things that don’t mesh with how you think things will work but you should NOT be designing the database.

2) Get developers to read code complete.

I think there are many other important things that we should be harping to improve the quality of custom software in the wild – things like an automated build process, instrumenting code, setting up a proper development environment/model, utilizing design guidelines, and utilizing test driven development. There are so many great concepts that developers should be using in all virtually all applications; given that, I don’t see how I can push distributed programming down all of their throats. Honestly, distributed computing falls pretty low on my list of wishes for developers because I believe that it will have a dramatically smaller effect on the big picture of software development when compared with all of these fundamentally important aspects of the software development lifecycle. Then again, maybe it’s just me that is clueless.

Comments (8)

  1. Ferris Beuller says:

    It should be called "Feature complete" not code complete. If it was called code complete there would be no need for QFE would there now.

  2. Ferris Beuller says:

    Thats nice but thats now how small companies work. People take on many roles in small companies so you cant say, oh well will have a DB specalist here doing JUST and ONLY the database and this person doing this… that.. the other…

  3. On preaching to the choir – amen.

    Small companies DO have a problem, though – developers usually don’t have the time, or management buy-in, to improve their techniques. The "work-till-its-not-broken" mindset is quite common, followed by a move to the next project that is broken, and repeat.

    The thing is that this process can hold together for quite a while. But then usually success (hooray!) increases the load on the systems, which then buckle under the weight (uh-oh). That’s when the scramble starts.

    I see no true solution in these cases other than educating management. This is, at best, career-threatening ( "well, Jim over there doesn’t seem to be having the problems that YOU are raising." ) at worst, well, the same.

  4. Take Outs: The Digital Doggy Bag of Blog Bits for 24 February 2004

  5. Ferris Beuller says:

    Small companies do not have the resources (and I do not include people as resources as thats another topic I can rant on all day and night), or finance to have a 1:1 type mapping for technology : skilled people.

    If something needs done it gets done by whoever can do it. You get your hands dirty in a small company. Plain and simple.

    They are more "preactical" as in whatever works does the job. They go through more redesigns of products as they tend to get more addons and patches and fixes of kludges but then again ususally the products are not complex ones so its not so much of a problem.

    Then you have companies that are in R&D and doing the next big thing, they are holding out as long as possible until the market for the product takes hold after trials, testing etc with industries. These ones usually go through periods of downsizing alot and more and more people take on many roles as they hang on as the market appears. Some go poof some just scrape by and then start to increase in again.

    At the end of the day its whatever puts the bread on the table and whatever gets the sale and works. Sure it may not be messy but thats the way it is.

  6. Alex Lowe says:

    I understand the "small company dilemma" because I have seen it/lived it. While I think the goal there is harder to achieve, it is still not impossible and really it is not even all that extravagant.

    I mean, surely even small companies can contract in database expertise. Will something like this happen overnight? No. If you are a responsible developer and/or manager and you read this then you should work towards a goal of having database expertise that you can call on when you need it (whether that means a full time DBA or a contractor). As I know you will agree, the money spent now on the expertise will be dwarfed in comparison by the money spent on lost productivity later.