Why unit testing in Visual Studio Team System – Again

I know there is nothing that I can say that will be viewed objectively, because I am a Microsoft employee and I have been tangentially involved with the VS Team System development team (I am the development lead for the patterns & practices). That said, I would like to dip my toe in the fire of “Why unit testing in Visual Studio Team System” which I translate into “Why unit testing in Visual Studio Team System and not just plain Visual Studio”. For background information see the following posts:

There are a number of questions that come to mind for me with this issue. Why is unit testing singled out? What about the other tools, like source code control, and static analysis, profiling, etc? Certainly these tools are valuable during development. So, that said why focus just on unit testing? Since nobody answered, I will jump in. J  

I think focusing on an individual tool clouds the issue. Certainly if we are talking about one and only one tool then I don’t think there would be any argument, it should be included. If we are talking about more than one tool, which in the case of Visual Studio Team System then it gets more complicated. In many cases I can solve an individual tool problem with an existing tool. For example, I can solve the unit testing problem today with NUnit and NUnit Add-in. If I want to do static analysis I can use FxCop. If I want to do source code control I can use any number of tools (Clearcase, StarTeam, Perforce, etc.). The problem that I have is putting the whole thing together and getting some sort of consistent experience. In my consulting work I would often have clients who strung together their development environment with a myriad of scripts and various versions of tools, some built in-house, some purchased, etc. The first thing that came to mind was how many people would use these company’s products if they saw the mess that was used to produce them. The closest analogy that I can think of is; you would not eat at a restaurant if you saw the food being prepared.  That said it is critically important that these tools (unit testing, static analysis, profiling, source code control) be used during development because they have shown productivity and quality improvements when used individually. Visual Studio Team System provides an environment which enables a number of development tools to be used and the user can get a consistent experience from implementation to implementation. I don’t have to write a custom task for my unit testing tool in MSBuild, I can use the one provided. I don’t have to figure out some goofy directory hierarchy so I can share my FxCop rules amongst a team of developers. I can easily run my unit tests locally on my development machine and then if someone in the QA department needs to run them they can do it to without having to have a custom development setup. I could go on and I probably should but it is for problems like these that Visual Studio Team System was developed and I believe gets to the root of why these tools are not adopted in a broad sense.

Of course this is my opinion and in Dennis Miller’s words “I could be wrong”. If you think I am off base please feel free to let me know.

Comments (24)

  1. Russ C. says:

    That’s a fair point.

    TBH the only thing I most want in the next version is the ability to set default views for types in Quick Watch , so if I can tell Visual Studio to always do a ‘ToString()’ when I QuickWatch a Guid.

    Oh and better support for Dual Monitors 😀

  2. Ian Cooper says:

    I would suggest the outcry relates to the growing awareness and popularity of Test Driven Development. As TDD grows a unit test tool is becoming as essential as a debugger to the practice of developing software. Coverage tools, static code analysis etc are all powerful adjuncts to this feature, but they are used to review development not as part of the development process. It is great that Microsoft has embraced TDD and is adding first-class support in Visual Studio, but I suspect for many the decision to leave unit testing out feels like leaving the debugger out. TDD is not a ‘team’ issue. Its just as valuable to the single developer. And while developers can use NUnit instead, it seems strange to need to switch tools between individual and team environments.

  3. James says:

    I think the plug-in architecture should rule, and that the set of Team System tools should be optional components to a single instance of Visual Studio. It’s fine to deploy them separately, but make it so that developers and testers alike can utilize them in their respective environments. The analogy I would use is Sourcesafe. There is a standalone client, but it also integrates into VS.NET perfectly.

  4. Jeff Clark says:

    Unit testing is singled out just because it’s hot, simple as that. However, it should NOT be singled out. The same argument applies to static analysis, code coverage and profiling tools. I use them all the time during development. How do you know how well your testing or unit tests cover your code without a code coverage tool? Whenever I finish a significant part of code I do a quick profile just to make sure I didn’t do something dumb. (I save the in-depth stuff until later in the cycle when the code settles down.) FxCop gets run pretty much every time I compile.

    To me it comes down to Microsoft’s goal. Are they just shoving TDD, profiling, coverage, etc bullet points on a list so they can say "yeah we do that" or do they really believe in this stuff and they want developers writing code for their OS’s to write the best code they can? If they are just after the bullet points then well goody they’ve succeeded. If they really believe in this stuff and want us to write the best code possible for their OS’s then it makes no sense to force everyone to use Team System. If they do who will be left using Visual Studio? I suppose just those who use 3rd party tools and those who aren’t serious developers.

    Visual Studio should contain what a single developer (who may or may not be part of a larger team) needs to develop code well. Team System should contain what a team of developers need to work well together.

  5. Ian Cooper says:

    Jeff hits the nail on the head. What’s ‘Team’ about these components. They are part of a professional developers toolkit whether they are working alone or as part of a team. That’s why they have been singled out. Editions of Visual Studio without these products will seem stunted.

  6. Unit testing is singled out because, uniquely, you get locked in to whichever unit testing tool you start with.

    If you have Plain Old VS, you can (I assume!) move easily from VSS to the source control in Team System; you can move easily from FxCop to the analysis tool in Team System; but you simply can’t move easily from NUnit to Team System’s unit testing system.

    With everything except for unit testing, you can start with whatever you want and upgrade/switch to Team System later with minimal difficulty. With unit testing, you can’t make that switch. That’s why unit testing should be in the base version.

  7. Roy Dictus says:

    I think unit testing should be part of VS.NET because of the Red/Green/Refactor cycle. Nobody seems to wonder why there is refactoring support in VS.NET, so why wonder about unit testing support?

    Of course, unit test execution is also part of a good automated build process, but unit test creation is just part of writing code, IMHO…

  8. Because Unit Testing is the present-day equivalent of Debug.Assert.

  9. The reason I think unit testing should be made available is the exact reason Sam Gentile stated. Unit Tests are a developer’s task to create. They’re part of the development cycle.

  10. Jeff Clark says:

    Aaron maybe you can help me understand because I don’t understand this viewpoint. Is a poorly written unit test that only covers 10% of the code really valuable (other than being able to claim unit test have been created)? Isn’t code coverage needed to determine that the unit tests cover all/most of the code? Doesn’t static analysis help us identify areas where we may not have accounted for something in our code? I run FxCop after every compile so it’s an integral part of my developent cycle. Doesn’t we need to profiler our code at some point to know that it’s written optimally?

    What about a team consisting of one person? Are we saying that a one person team needs to unit test, but doesn’t need to profile, run code coverage or do static analysis? If you’re a team of 2 you still don’t really need the "team" version but you still need to do these functions.

    If you (or anyone else) can point out what I’m missing I’d appreciate it.

  11. I’ve only been using Unit Testing for a short while now, but it’s already starting to feel natural. It *is* a part of the development process, should be used early and often. (Before the first line of code) Retrofitting Unit Tests after the fact would be an absolute nightmare.

    My question is similar to some of the others. Why so many versions of Visual Studio anyway?

  12. Andy Lawrence says:

    Unit Tests form and integral part of the codebase and are tightly coupled to it.

    The (best?) practice of Test First see unit tests as to driving design rather than simply a testing an implementation. It leads to the well known cycle of: Test (design), Code, Refactor.

    VS will have the last two parts – Code, obviously, and Refactoring support – in all versions, why not the first?

  13. The only comment I’ll make is that I have seen a way to *almost* seamlessly reuse NUnit tests as Team System Unit Tests (I embarrassed to admit I don’t know the official terminology for our integrated unit testing).

    So, while the ‘lockin’ comment is true in the general case, I think it might not be that cut-and-dry in the NUnit/TeamSystem case in particular.

  14. As promised, here’s a small primer on testing. Note: if you’re an inveterate tabbed-browsing researcher like me, hit up my testing linkblog page for hot hyperlink action with all the sites mentioned herein. The best introduction to testing in general,…