10 years on – is software development getting any better?


The last 10 years have been exciting if you are a developer. Lots of new programming languages, lots of new IDEs, lots of new tools, lots of new approaches, lots of new apis but … as an industry are we developing software any better than we were 10 years back?I have been at Microsoft for 12 years and have been lucky enough to see the work of lots of different teams in different companies during those 12 years and I am struck by one rather scary thought. Things are not getting any better. Sure – there are great teams out there doing amazing stuff – but there were great teams doing amazing stuff 10 years back (and 20 and…). It is the “norm” that concerns me. I think the norm is no better than it was 10 years back, arguably worse (I will expand on this another time). What do you think?

Comments (26)

  1. chris seary says:

    Hi

    I don’t think that your question covers the issue properly. You have the option ‘No real change – there are good developers and bad developers’. However, development is a process, involving many people.

    Agile development is only successful if the team and the client define and map requirements efficiently. If it does not, then agile development usually fails. I’d say this is the most important aspect of delivering software currently, and developers are only a part of it.

    Better tools will mean greater prdouctivity in the coding part of the development cycle, but then so does typing more quickly ( 🙂 I’m being a little facetious here). I’d say that a properly architected solution, with a good plan for delivery of requirements, great build team and efficient testing will all work together to give a better product.

    Only my humble opinion   🙂

  2. Hi chris – howw are you doing?

    I think development is still very much about the quality of the people. In that sense, tools, technologies and process have failed us. In effect I believe we have failed to raise the abstraction layer significantly over those 10 years vs the previous (when we got away from assembly, coding UI (draw x,y stuff) and procedural languages. Hence the quality of the individual still is extremely significant.

    I think the journey we should be on is one of ever increasing levels of abstraction in sw development. Not sure we have actually done that in those 10 years – although clearly there are examples where we have such as the CLR managing memory for me.

  3. chris seary says:

    Hi Eric

    All’s fine down in sunny (often rainy) Gloucestershire! Hope you had a great holiday.

    I certainly think that people are important, but the whole team rather than the developers. In particular, focusing on requirements, and being able to map this through to testing and validation (to which Team Server contributes greatly!!!!).

    One thing that I see happening at present, and maybe this is my incorrect perception of abstraction, but so much new effort is going into components that don’t add a bean to the end product as regards requirements.

    Some new additions to C#, for instance, are simply compiler tricks that actually compile the same as .Net 1.0. There are also many templates available that model design patterns which sometimes don’t add to the system requirements.

    This happens in all aspects of software – for instance, you may remember three years ago that many applications were using WS-Security because it was the flavour of the month, when really SSL was often more appropriate. I’m certainly a big fan of WS-Security, but only where the requirements dictate it should be used.

    The beauty of .Net when it first appeared was that it mapped directly to use cases. For instance:

    -user enters personal details

    -personal details incorrect

    -user re-enters personal details

    This maps directly to adding:

    text boxes;

    validation controls;

    buttons

    to a web form. Minimal coding, massive satisfying of requirements with just a little drag and drop.

    This is what I think made .Net so popular, and what makes it more productive than most other environments.

    But what does the MVC add to many of the  applications that are using it? There are certainly valuable uses for it in the appropriate cicumstances. However, it’s being used very often now where it doesn’t relate to what needs to be delivered. There are lots of developers who talk about this being ‘the correct way to do it’. But that’s not true. The only thing that is correct is the requirements. Your resources dictate how you’re going to meet those requirements.

    In my humble opinion, I think that we’re losing sight of these efficiencies, and perhaps we should think of ourselves as Software Engineers rather than developers – ie. everything that is done is done for a reason. No ‘cool stuff’ unless it satisfies what is needed in the system.

  4. Jonathan Allen says:

    Can you have build the stuff you are building today using only ten-year old techniques and tools?

    I can’t, at least not in the time frames people expect of me. Therefore I have to say software development is getting better.

  5. Jim says:

    Having started my prgramming career 30 years (yawn) ago before Microsoft, let alone 10 years, to me it seems that software is merely being laid on more software in an ever more involved and esoteric fashion swallowing huge mouthfuls of memory.

    A statement (from the tutorial) such as:

    Me.m_ListDetailsPart.Dock = DockStyle.Fill

    seems so abtuse that I have to read the whole program to grasp the purpose of this kind of statement or is all programming now aimed at website interfaces?

    I congratulate you who manage to create all this software.  It is a pity that the poor PC’s registry, memory and hard disc fill up that every 3 years we have to buy yet another PC to run the latest software that does virtually as the same as the previous versions but just slightly differently.  Yes, someone seems to love moving menus and drop downs around to confuse the innocent among us.

  6. Heretic says:

    About 10 years ago, I worked on a system with around 500 users split over three sites. The database was of the order of 50GB (a  Dual 100MHz pentium Compaq server), and the clients were written in VB4: on 8MB PCs we had sub-second response times.

    My current project involves a database of around 100-150GB (quad XEON procs, 4GB RAM)

    a number of separate mid tier servers (quad XEON yada yada) and our clients are WPF – so we’re talking game spec PCs…

    We are struggling to support 300 users per server…The arrogance of the developers is the only constant – “This is the RIGHT / ONLY way to do it – a Classic three tier design, using MVC”.

    Yes hardware is getting cheaper, Yes we have gradient fills, but all the customer wants is a product that works. They dont care about the purity of the architecture. They want bug free, responsive software.

  7. JayB says:

    Hmm? I have to agree in part with Jim and Heretic. Languages are getting so high level that the developers don’t need low level knowledge and are becoming disconnected from the real issues involved in development. Timescales are always an issue, but a good developer can manage that.

    I write c# code (because it’s my job) but I don’t really need a managed environment – I can manage my own memory thank you – I never had a problem (i.e. leaks) with c++. At least then I knew exactly what my program was doing and good God I love pointers 🙂

    Ho Hum – time moves on, and so do we!

  8. JayB says:

    Oh, and a final thought… Funny we’re asking the question (and giving it thought)?

    That should be an answer in itself?

  9. Paul Eden says:

    This is a knee-jerk reaction and I may reply a second time later, but I take exception to the third option of "no change – still good and bad developers".  I consider myself to be adequate – I’m sure I could still lear far more than I currently know, even after being in the business for nearly 7 years now.  

    This option makes it sound like development and the development process is soley impacted by developers.  I’m sure that in a Software Engineering Degree course (which I do no have behind me) people are taught that the developer is the centre of all code creation and any deviation from accepted ethics will reward you with an eternity of torment in hell.  

    In a perfect world I’m sure this would be the case, but development is business and while there is quest for profit the developer will never have the final say in how things are done; to flag those at the mercy of Sales or customer set deadlines as "bad" developers is quite unfair.

    I would prefer to have seen this option as " no change – businesses still own the project life cycle"

  10. Cuban says:

    I’d love to say that things have improved but what you find with software and technology in general is that as you make one thing better something new comes out that makes it more difficult to get right, so there is a continual fight to keep development processes up with development tools and features e.g Ajax. Ajax has brought forward more usable and quicker web interfaces but with it come harder ways to test and debug.

  11. Phild says:

    In some ways yes but no in the nuts and bolts way. Tools and environments have moved too fast for most developers to keep up with. Almost everyday there is a ‘new’ way to do something that was only just being understood using the old way. A developer could spend all working hours and most of their fun hours trying to keep up with new stuff and working out what to use and what not to use.

    I like new toys and progress but when the job just has to be done tried and tested is best.

  12. Graham says:

    "Lots of new programming languages, lots of new IDEs, lots of new tools, lots of new approaches, lots of new apis".

    Not necessarily well developed themselves, not necessarily improving productivity (and in some cases, reducing it) and with thousands of properties, methods and classes, not only can it take hours to find the "few lines" to achieve something simple, there are lots of opportunities for error.

    The development environment is more complicated than ever. It needs to be simplified to see development improvements.

  13. Rub says:

    I think we should highlight the difference between proper programming and being able to write software in a particular language. Being able to kick a ball doesn’t make you a footballer.

    New tools will lower the bar so that pretty much anyone can write software, produce nice GUIs, etc.

    This is very good for businesses, because it makes software developers affordable.

    Still, because these people are not properly trained (when you see code passing a big array of data by value as opposed to by reference you understand my point) it makes the results they produce pretty poor in terms of quality.

    So good software development comes out of lots of practice, obviously, but also some formal training to build on to.

    Therefore there’s pretty much as many good developers out there as there were 10 years ago, only now there’s bags of poor developers out there.

  14. Nick H. says:

    Hi Eric,

    When you were in Yorkshire, did you go to the Railway Museum? There’s a train there called the Mallard, and it remindes be a lot of current software development.

    Mallard set the world steam speed record, and was steam engineering refined to an art. Now it’s stationary, in a museum, eclipsed by a better way of doing things.

    I think the last ten years have seen imperative and object-oriented programming becoming ever more refined, but the realisation must set in at some point that a better way is needed.

    Alternative paradigms (declarative, functional) maybe that alternative. Maybe not. Either way, any back-slapping about how good things are now is just distracting us from looking for better alternatives.

  15. John Price says:

    I think the problem at the moment is that with all these really nice tools, software development is too forgiving. There are so many programmers out there that have the attitude of if it happens to work it must be right. Theres little thought, generally, given to things like, alternative browsers, security, scalability, performance etc.

    With hardware costs coming down, more and more of these issues are masked.

    With more and more people taking a crash course in programming (I watched an advert on TV last night that suggested that you took their 28 day course and then moved into IT), theres less emphasis on building something solid and more on get it out quick (my management here some of the ‘time saving’ features in IDES these days and think they can halve delivery times), The number of times I have heard comments like ‘we dont need a spec – we are doing agile development’ to mean we dont know what we want but when you build it we’ll tell you what we wanted different!

  16. Vince says:

    10 years ago, i was developing State-full LAN applications.  Today it is State-less web applications.  The Stateless applications are much harder and bring more pronounced headaches in terms of scalability, performance and security.  Also, 10 years ago, we had a raft of developer friendly 4GL’s, most of which died in the face of the Internet.  (Powerbuilder being a rare survivor).

    So, i think the runtime environment has become far more complicated and the design and development tools, although improved, have in no way compemsated for the shift in environment.

    Some of the ‘gap’ is being filled by developer brainpower, but the remainer of the gap is showing up as a drop in overall developer productivity.

    10 years ago, it was much easier!

    Pay rises all round?

  17. AlecM says:

    I think there’s no doubt that what we are capable of producing with today’s tools has definitely improved, but the amount of time available for learning the best way of achieving this, along with the increased expectations of users and shorter delivery time scales is a bad mix that can lead to problems.

    Getting recognition from the project/feature stakeholders that what they’ve asked for will take X amount of time rather than the Y they demand is an age-old issue.

    Having stakeholders that are able (and willing to make time) to explain what they actually need is a large factor (as well as the developer’s ability to understand what is presented) – poor communication will lead to poorly realised results.

  18. Andy Burns says:

    New approaches? New tools? New technologies? Hmm. It sort of misses the point – most of what is bad about software starts long before development. I’m thinking of:

    – Poor requirements gathering

    – Customer not reading (or understanding) documentation – and UML doesn’t help that!

    – Badly set expectations at sales time

    – ‘Lowest cost wins’ vendor selection results in management wanting to ‘skip’ minor things, like requirements, prototypes, even Specification.

    – Scope creep.

    I’d also point to increasing technology churn – I can’t be the only developer wishing that Microsoft would stop introducing new versions of everything that don’t really help me achieve more. Heck, I’d like them to stop releases that would let me achieve more – I need to catch up first. WPF, WF, WCF, SilverLight, .NET 3 – I’m considering giving up sleep.

    To me, the critical problem is people’s expectation of cost. They expect, say, a building to be expensive – but software to be cheap. Hence, buildings (are usually) properly designed, analysed, tested and built – and are there in 20 years time. Software, well, isn’t.

    Sadly, ‘Software Engineering’ is an oxymoron.

    And as to the point about the customer not wanting ‘pure’ architecture, I see this from the other end, being asked to build the quick, dirty, impure but ‘cheap’ solution. So I build it – and then the requirements change, and changing what I’ve written suddenly becomes a nightmare.

    It’s a trade-off – but you should always attempt purity, and make sure the customer understands the consequences for the future if you depart from it. For example, with MVC that might be being stuck with a particular user interface – but that might be a trade you’re willing to make.

  19. alan arnott says:

    good code is good code whether it is written in .net or cobol. good code is what delivers effective service to users and organizations.

    package software has improved massively in reliability in the last 10 years. however, it has not always imrpoved in quality.

    any solution based on an "ism" is prone to issues. oo, agile etc are only as good as the practitioners.

    richer interfaces and sophisticated data retrieval mechanisms can deliver significant benefits but come at a performance cost. there is no general rule – each project and system has to be assessed individually. for some procedures, paper and pencil is still the most effective.

    in general, it seems to me that many developers and their sales staff have forgotten the feature/benefit paradigm. if it isn’t a benefit, don’t use it.

  20. Hi, I’m totally new to all this stuff as I’m still a Newbee. So far as the subject under discussion is concerned, I’m in two minds about this.

    On the one hand, 10 years ago the only way I could afford to program was with a second-hand Acorn Electron from the local charity shop. PCs were expensive, internet connections were expensive, and software was expensive. Being unemployed, PC programming was completely beyond my budget.

    However, programming the Electron was so comparitivly easy compared to today. All you needed was some small books on BASIC, assembly language, firmware, hardware and miscellaneous books on graphics, sound and games programming (All of which could be obtained from the charity shops for pennies). The programming interface was quick and simple to learn, you could have a simple HELLO WORLD program up and running in under five minutes and you didn’t have to practically get a degree in software development before you can even begin to have fun programming the machine.

    Now I have access to an Athlon 64 4000+ with an ATI Radeon HD 2600 Pro and a cheap 8MBit Unlimited broadband connection with Windows XP Pro and Visual Studio 2008 Express, along with all kinds of great freeware development tools available online.

    All the above gear can be bought very cheaply from places like Amazon or eBay along with full-on development software and tools available either for free (Visual Studio 2008 Express and Truespace 7 being great examples of great free development tools) or very cheaply, again from eBay or Amazon.

    Unlike the situation 10 years ago where software was only free to distribute as source code while a licence to distribute compiled software costed £300+ using commercial development tools, now it’s all free!

    Being unemployed (again), all the above adds up to me being able to afford to join the party, along with thousands like me.

    Unfortunately, the party isn’t that much fun for me at the moment. The first time I opened up Visual Studio, I saw a hideously complex interface that left me with a distinct WTF feeling and without a clue how to even get started.

    At least with Borland’s Turbo C++ interface for DOS, which I tried on a Celeron 400 I had a short time before, it was faily easy to get started as the IDE was so much simpler and I could get on with coding almost straight away.

    However, with help from some books from the library and help available on the internet, I’m gradually learning how to get the thing to actually DO something.

    With free add-ons to VS 2008 Express like DirectX and DarkGDK SDKs the rewards are potentially very great. I’ve already seen several examples of programs produced by this software that look absolutely fantastic and light years away from stuff I could do on the old Acorn Electron or even the more recent Celeron 400 with DOS. So it’s definately worth putting in the effort to me learn how to do this stuff too.

    However, I’m still left with the distinct feeling that I’m in really need of an expensive Software Engineering degree to get the best out of modern development tools today. Not fun.

  21. Steve Jones says:

    Darren – Some suggestions, which I hope might help. (Apologies to other people for this being a bit off-subject)

    Start a console application using VC++ Express, something nice and simple, but it will hopefully convince you that it’s just like Turbo C++ but better.

    Then try the same with Visual C#. Start with Hello World. There’s a lot there, but hopefully you will find that it’s initially a bit like C++ but more complex, after a while you should find that it’s actually more straightforward, especially as you don’t need to really know about pointers etc (at least not yet).  There’s lots of help, tutorials etc out there on the web. Then try a simple Win Forms application, put on a button and a label, double click the button to get its click event and just change the text of the label. Then try a few ideas of your own.  Don’t try WPF yet as it will put you off for life (you can come back to it much later, it’s really good but not for a beginner).

    Once you get the hang of it, you could try the XNA Game Studio Express, for which there are many free tutorials. Think of it as being what you’ve done in the past, with a bit more to learn up front but rich rewards afterwards.

    Hope this helps,

    Steve

  22. Wow – thanks all for taking the time to write. Lots for me to mull over.

    Interestingly it looks to me (from the votes) that we don’t as a community believe things have improved. I am pretty confident that if we had the done the same poll 10 years back, we would have given a clean thumbs up to the improvement in the quality of sw being developed.

    Oh and Steve – tx for taking the time to answer Darren. You are right Darren – it was MUCH easier to get started in those days. We are aware and we are thinking through stuff to improve this… fingers X’d.

  23. Just saying a quick thank you to Steve for his advice.

    Every little bit really helps for the average newbee feeling-like-a-total-dumba** programmer like me.

  24. Steven Gill says:

    I still think there is lots of improvement potential in tools, why can’t the IDE understand at a high level what you are trying to write and suggest ways of doing it and what still needs to be done?

    Some ideas decreasing in complexity:

    1/ It would be nice to be able to have a “conversation” with it like I would another developer to get across the purpose of the software, the things it really needs to do well and those that I’m not too worried about, how to represent things and store them, get it to look up appropriate code samples and tune them to the specific use.

    2/ pseudo-code parser, to nail down the essence of the program and create a structure in your chosen language and technologies then monitor what you program to ensure it fits the pseudo code, dare I say a Clippy for programming but not in a rigid completely templated style.

    3/ Analysing comments and providing suggestions, if I start a method and say for example “\ Save options to registry” show code snippets for accessing the registry, ask how and where you want to store them, guess what you want to store from parameters and from the rest of the program and tweak the snippet to implement it.

    This relies a lot on AI otherwise it will seem forced and templated, it needs to feel like getting suggestions from an expert developer in the machine.  

    For me I spend far too much time looking (mostly codeproject.com) to see how other people implement particular functionality or looking for frameworks (to wrap complex behaviour such as Bluetooth or multitouch) and graft it into my software as writing code is not my day job, why can’t the IDE do that?

  25. M4rky says:

    I don’t think there is any change. We now have a great set of tools and languages to play with but the complexity of software is greater yet the timescales given to develop these complex apps are smaller.

    The time needed to develop a piece of software Programming has always been very difficult to quantify. Often you can double your original estimate and still be way off what it really takes.

    How often do developers ignore good code for the quick solution to meet deadlines?