Carrying around the weight of the past


I’ve been having an incredibly interesting email conversation with Michail Arkhipov (a Dev lead on the Web.Net) team and I’m pretty torn on how I feel about it.  A little background might be a little helpful to give some context to the discussion.


There is a whole heck of a lot of lines of source code inside VS and MS as a whole.  Millions upon millions of them.  It’s taken as a given that for the company to be successful, developers need to be productive, and for developers to be productive, managing all that source and the process of producing the binaries from the source must happen very smoothly (in a perfect world transparently).  In order to facilitate this there are full time employees and teams out there that work to make that happen, and to make sure that the “build” is always getting produced day in and day out.  This is not an easy job to accomplish.  There are teams who are actively working on one part of VS (like our team and the C# component) and there are teams that need to use the work we produce in order to do their own work.  For example, we code some of work in C++ and so we need a working C++ compiler in order to function.  A lot of the Team System code is written in C# and so they need our code to work.  Then there are large teams like the runtime whose work ends up impacting everyone.  When those changes are finally batched up and applied to our branch it ends up conflicting with the changes we’ve been making.  Those conflicts need to be identified, resolved, and propagated out so that the build can proceed.  


In order to make this possible all (or at least much) of MS is standardized on a common build system we call NTBuild.  I don’t know the history of it, but judging by the name, I assume it came about when they were building NT (so potentially nearly 10 years old).  The system has many flaws and faults, but it works and there’s a lot to be said for that.  Unfortunately, there’s an issue for our team when it comes to this.  Specifically, C# projects now work with the next-gen build system that MS has (appropriately called MSBuild).  In order for things like C# intellisense to work you need to have one of these projects set up.  This is a concern for other teams inside MS because it means they now have to support two build systems.  The NTBuild system that the entire organization uses (and which is supported by a full time staff), and the the MSBuild system which they would need to take care of themselves. 


Because of this Mikhail was requesting that our team do work to support source files that aren’t in a project.  He raised many very good points (and hopefully I’ll be able to present them here).  The primary issue was that if we did not do this we would end up with teams inside MS not being able to dogfood our product because they would lose so much productivity trying to support this sort of dual system.  I can appreciate that concern.  However, it raised several issues for me, including:



  • Are we solving a purely Microsoft problem here?  Do our customers benefit from this?  Would they benefit more from us working on something else?
  • If we can’t get teams internal to switch over to this new terrific build system which we want our customers to use, then how can we be sure that it really is going to be good enough?
  • Are we doing ourselves more harm by sticking with the existing ancient system?  Should we be moving to newer and better ways to handle this process?

There’s a lot of information about MSBuild out there, so I won’t repeat it.  However, there is one really nice thing about it’s integration in VS2005.  Specifically, when you do a build from within the IDE you get the same build system as when you run MSBuild externally on the project file.  So we have a system now that will allow you to build internally and run from your existing build system without any changes, and which will produce the exact same results.


So what do you think?  Is it worth adding that feature that will allow C# to work outside of a project?  In your own organizations (or on your own) do you find yourself wishing for this feature?  Or are you ok with the support we currently have.  If you’re not ok, would the new MSBuild system solve your problems, or do you need support that goes far beyond that.  Do you think that this is just an MS issue and that we should fix our internal processes rather than adding more complexity to an already complex tool?


If you are looking for more then let us know what your build systems are like so that if we do this we can try to make sure that this won’t be solving a purely MS problem but also yours as well.


Comments (8)

  1. Ziv Caspi says:

    Let me be blunt — if you don’t, I don’t think the larger projects who actually ship will switch over.

    In my team (we build with build, a.k.a. NTBuild), as many people use Source Insight or Visual Slick Edit as there are Visual Studio .NET 2003 users. In a poll we’ve recently conducted, some VS.NET users reported writing code with VS.NET, but reading code with SI or Slick. The "all the C/C++/C# files at this directory structure belong to my project" model of these tools is a winner compared with VS.NET. The tagging ("browse info") that comes along with this model is the real killer.

    VS.NET is behind when it comes to handling huge projects compared with either Slick or SI. Unless you provide a tagging system which is at least as good as theirs, without imposing on us switching to your build environment (*), you won’t win our hearts.

    (*) Yes, I do realize MSBuild is probably far better than NTBuild. But when you face porting a >1M LOC project to a different build model, you really need to be a lot better. A better IDE is not considered a good argument for changing to another build system, particularly when we already have reasonable IDEs deployed.

  2. ZIv: Thanks for letting me know. It’s crucial that we understand how developers are going to actually use our tools. If a significant number of them would benefit from this then.

    I’m glad to find out that there are people out there that have systems like ours. It means this isn’t something that solves just a problem for us and no one else. Now I just want to know how important this is to teams out there in the rest of the world.

    BTW. Is there any way we could see the results of that survey?

    Also, when you do open files outside of a project, what are you mainly doing?

    Browsing, viewing definitions, reviewing, simple editing, full out editing?

    What do you find unpleasant about this experience now?

    THanks!

  3. Peter Ibbotson says:

    I hope to catch up with Alex Kipman at TechEd and run through some of my thoughts in this area. I just need to get my act together (in terms of documenting my project structure) before I meet him.

  4. Personally, I’d rather you were dogfooding MSBuild.