Code metrics from the command line


Yesterday, we made available a power tool that allows you to analyze assemblies in order to understand the complexity metrics associated with those assemblies. I’m going to describe how to use that power tool and the results it generates.

If you are not familiar with the code metrics capability built into Visual Studio, please see this MSDN documentation for details. This power tool uses the exact same functionality that you see inside the IDE, but encapsulates the code required to do the actual analysis in the metrics.exe application, described below.

Installation

Grab the power tool from this location.

After successfully installing the power tool, you will find two new files in your (%PROGRAMFILES%)\Microsoft Visual Studio 10.0\Team Tools\Static Analysis Tools\FxCop directory. One is the actual executable itself, Metrics.exe, the other MetricsReport.xsd, the XML Schema that defines the results format Metrics.exe generates.

I point this out as we don’t make that very obvious during install. We don’t slam anything into your Windows Start Menu, drop desktop icons, etc., so be aware of this directory location and executable as you start to play with this power tool.

Usage

To quickly see what this little power tool can do, locate any assembly you would like to generate code metrics and use the /f and /o command line options like so:

metrics /f:Metrics.exe /o:MetricsResults.xml

This will actually run the analysis over the metrics.exe itself and push the results into a file in the same directory called “MetricsResults.xml”. I point this out as you can run metrics analysis over any managed assembly whether you have the code for that assembly or not. However, if you do have the PDBs associated with those assemblies, that’s best. ( If you don’t have the PDBs you will get warning CA0068 ).

The /f and /o combination is the simplest form of usage, but many times, you need to provide the tool a little more information, such as where to find dependent assemblies. The /directory or /d option instructs the tool to look in the current directory to resolve necessary dependencies, while the /searchgac or /gac option will allow the tool to search the GAC to resolve those dependencies. You can also indicate per-file dependencies with the /reference or /ref option.

Results

But what do the results look like, and how do you consume them? Well, I would love to be able to say to you that the results are automatically pushed into your Team Foundation Server data warehouse or reported seemlessly in your reports directory of your team sharepoint projects, but alas, that is *not* the case. The results are XML in the form defined by the MetricsReport.xsd schema file. This is nice, as it allows you to consume and process these results any which way you’d like, but you do need to understand the form of the results to do so.

Double click on the MetricsReport.xsd file and it will appear in Visual Studio, like so:

SNAGHTML137bfcb

If you click the  “Show Graph View” and “Top to Bottom” toolbar items, you’ll see the schema represented in a tree-like structure like so:

SNAGHTML13d2b88

This is a great way to quickly understand the containment structure of the results you will see output from this tool which will be important when processing the XML. However, it doesn’t quite give you all the information you need in regards to the actual values you can expect, especially when wondering what types of metrics will be reported. For that, there’s nothing better than inspecting actual data generated.

Here’s a snippet of some results created from my example above:

SNAGHTML85495

The elements I highlighted are the five metrics that we are tracking, and these five metrics will be child elements of the five containers found in the xsd. Target ( the actual file you specified in the command line ), Module ( the .NET assembly found by following the target ), Namespace, Type, and Member.

Why did we provide this power tool?

Many customers are looking for a way to generate code metrics information as part of their build process. In this way, the complexity associated with assemblies coming out of your nightly builds could potentially be tracked and examined over time, which is where I think information like this is most valuable.

With this power tool, we’ve provided the raw capability, but we have a lot more to do to get to automating other parts of the desired workflow, such as providing default windows workflow activities that include this capability. But for now, if you find this information valuable, you can certainly incorporate this into any MSBuild or Windows Workflow ( Team Build is WF enabled in TFS 2010, by the way ) process.

Enjoy!

Cameron


Comments (30)

  1. Awesome says:

    Thanks Cameron VS Ultimate Team.  We've been looking forward to this for awhile.

  2. Jakob Ehn says:

    This is certainly great news, we have been waiting for this tool for quite some time! 🙂

    I have written a blog post with a sample that shows how to run and integrate the results from the Metrics.exe inside a TFS 2010 Build Definition. The build process template and the custom activity is available for download. Check it out here:

    geekswithblogs.net/…/integrating-code-metrics-in-tfs-2010-build.aspx

    Cheers/Jakob

  3. Excellent Jakob! This is exactly what we were hoping for as we released this tool! Also, Buck has also shown some detail around a similar process as well which may be good for folks to see as another example. blogs.msdn.com/…/code-metrics-command-line-tool.aspx

    Cameron

  4. Krip says:

    This is fantastic – great work.  Anyone built the necessary XSL files for integration with CruiseControl.NET?

  5. Alfred Myers says:

    Are there any plans to push the data into the Team Foundation Server data warehouse?

  6. No immediate plans, but this is something we very much want to get to.

  7. Daniel marbach says:

    It would be really great if you could also release fxcop power tools so that we woukdn't need to install vs2010 on our build machine! Thanks

    Daniel

  8. Bjorn says:

    Tried to process several assemblies at once with:

    set output=metrics_output.xml

    set metrics="E:Microsoft Visual Studio 10.0Team ToolsStatic Analysis ToolsFxCopMetrics.exe"

    @echo %date% %time% > %output%

    for /F "delims=|" %%i in (metrics_assemblies.txt) do %metrics% /f:%%i >> %output%

    Unfortunately the xml file only contains data from the last assembly in the text file

    I've run out of ideas here.

    Anyone?

  9. I'm not positive, but double check your for command. I think it may only be parsing the the first '|' delimited entry in your metrics_assemblies.txt file.

  10. also, you can list multiple /f options for metrics.exe, which will then process each of the files, pushing the results into the output file specified. You'll end up with multiple Target elements in the results file.

  11. Radu says:

    Why do you still create command line tools? A VS addin would have been more easy to use.

    After all, one must start this tool with funky params, move the files around on hdd, then go back to VS and see the results… seems like a half done job. Even if the tool is very promising, you and your team should focus more on User Experience – the easy of use part.

  12. @Radu, the functionality is already in the IDE. Right click on a solution and select "Calculate Code Metrics".

  13. Adrian Hesketh says:

    @CameronS – This functionality isn't include in Visual Studio 2010 Professional, it only ships with TFS versions.

  14. True enough. Code Metrics in the IDE is in the Premium and Ultimate SKUs.

  15. Michael Sparks says:

    Bjorn, The reson you are only getting the last assembly in the text file is that you are running the metrics tool for every assembly separately.  This will overwite the output every time.  This tool does not have an option like NDepend to read from a project xml file in which all the assemblies can be listed and consumed for the analysis.

  16. john says:

    Cameron, thanks!

    This tool is GREAT.

    I don't agree with Radu. i think this tool SHOULD be command-line oriented — it was the right decision.

    This enables an excellent build integration / CI story.

  17. Brandon Dimperio says:

    It appears that if you don't have all the references needed for a single assembly in your list to analyze… the xml writer won't write anything at all to the output file. Is there a way to tell it not to just give up because of one dll?

  18. Sahas says:

    Is there any custom activity to read the results file and show that in a some tree format (like how its shown within VS)?

    Thanks!!

  19. @Sahas I assume you mean a custom WF activity that you could use to incorporate into your Team Build process? Not that I'm aware of…

  20. bakopanos costas says:

    It would be very nice if the tool can run only against specific metrics… e.g. only lines of code. This would make it easier to resolve missing dependencies issues needed probably not by all metrics.

  21. Matthias says:

    I created an extension for Visual Studio,which allows to integrate the commandline-tool into the IDE; the download can be found at: http://bit.ly/iipJuw

  22. David Keaveny says:

    I agree with @john – this absolutely needs to be a command-line tool (VS integration with an add-in/extension is always welcome, of course); @Radu, making it such allows us to easily integrate it into an automated build process, without requiring Visual Studio to be installed on the build server. I've written a plug-in for the UppercuT build framework (code.google.com/…/detail) that demonstrates how insanely easy it is to include code metrics as part of each and every build.

    Bravo Microsoft, keep the tools coming!

  23. David Keaveny says:

    @CameronS, is there any chance you could give a little more detail on how to resolve dependencies? My build process (UppercuT) drops all build output, including dependencies, into a single folder (in the same way TFS does), but I'm getting a screenful of CA0060 and CA0058 warnings about dependencies, most of which live in the GAC.

  24. Rodolfo says:

    is it open source?

    can we have the code?

  25. No, it isn't. Binary only.

  26. Yaron says:

    Can I analyze cs files with it raher than full assemblies?

  27. No, this tool works against IL only, not source itself.

  28. John says:

    Is there any way to add a "filter" as a command line argument so it only writes to the output file what is returned in the filter of the results?

  29. 'Fraid not John. You'd have to post-process the resulting xml file…

  30. steveculshaw says:

    Image links seem to be broken   🙁

Skip to main content