DEV302 – Visual Studio Team System: Building Robust and Reliable Software

Presented by Eric Lee

(I'll post photos later)

Profiling Team Blog | Code Analysis Team Blog

Visual Studio 2005 Team System: Building Robust and Reliable Software white paper

- Microsoft has done a good job helping developers, but there's more we can do.
- Vision for Visual Studio Team System has always been productivity since first conceiving it years ago.
- Want to ensure out-of-the-box productivity on day one.

The "big picture of what Visual Studio Team System is trying to address
- Software dev today involves several disciplines (architects, developers, testers and project managers)
- Microsoft is introducing an integrated suite of features that everyone can use.
- Today, in this presentation, we're focusing on the developer role.

Going to cover in this session:
- Writing code
- Writing unit tests
- Reviewing work
- Run code analysis tools
- Prepare for operations

Microsoft Research technology and internal tool teams have supplemented Visual Studio internally (e.g., the profiler).
We've always asked if these technologies are ready for customers to use.

Write code and unit tests
- People use different process and methodologies

Jim Newkirk on Test Driven Development (TDD)
Poll: 50% know what TDD is; 10% use it

TDD tenets:
- never write single line of code unless you have a failing unit test; requirements expressed as tests
- eliminate duplication; constantly look for duplicated code

Test List
Task based
- 4-8 hr duration
- 15-20 min exercise at beginning
Brainstorm list of unit tests
Do not get hung up on completeness, can add later
Describes completion requirements

start >
  write a test for new capability >
    compile >
      fix compile errors >
      (thinking about code from usage perspective, not implementation)
        Run the test and see it fail>
          write code >
            run the test see it pass >
              refactor >
                next test (repeat cycle)

Demonstrated Red/Green/Refactor (creating a Stack Push/Pop class)
Use Assert class to make assertions about condition
Created TestClass with TestMethods first and then wrote enough code to make tests pass.
In TestMethods, calling methods that don't exist yet.
Can generate method stubs for non-existent methods; stubs throw NotImplementedException by default

Integrate testing into dev process
Testability is a primary motivation during development
Visual Studio Team System provides ideal environment for doing TDD

Back to Eric

Joined Microsoft as a tester
Poll: 40% familiar with code coverage; < 5% use it

Code Coverage
Explained what code coverage is
- Helps with predictability
- Effective testing
- Drives up quality
- Catch blind spots in VS


Used the Adventure Works sample application
Demonstrated code coverage using unit tests

Code Analysis
- Intermediate (compiling) In source code, can find things like bad coding conventions, outdated API use, etc.
- Executable phase (for .exe's), can use App Verifier and code profiling

Static Code Analysis addresses problems like:
- Bad code can still be syntactically correct
- Developers using different naming conventions
- Manual code reviews can be inconsistent

At Microsoft we still do a lot manual code reviews
We started building tools to search for things like buffer overruns (part of Windows 2000 security push)

At source stage, can look for bad names and obsolete API's

At intermediate stage, we see control flow
We can find more in-depth problems, like performance issues (e.g., string concatenations in a loop)

Default rules are based on experience, but the rules engine is extensible with custom rules

Demo of Static Analysis for managed code and C++

C++: First one is a classic buffer overflow problem; potential to exceed writable size of the buffer
Managed: DateTime.Parse(String s) does not explicitly provide IFormatProvider (internationalization error)
Managed: Using String.Concat instead of StringBuilder in a loop
Managed: Security issues replace array with a strongly typed collection

Static Analysis Benefits:
- Catch bugs at source
- Capable of complex analysis
- Leverage lessons learned
- Turn habit into rules

App Verifier addresses problems like:
- "It works on my machine"
-  "It worked before" (for example, on a prior OS)
- Run-time issues
- Some problems you can only find when you run your application

App Verifier finds problems as your application runs

As your application runs, App Verifier intercepts API calls for:
- Application Compatability (access registry in proper manner)
- Defects (not using handles properly)
- Security (using NULL DACL's)
- Windows Logo compliance (helps with finding problems with OS changes)

Demo App Verifier
- integrated into Visual Studio Team System
- originally shipped externally on Windows XP installation CD
- To run app under App Verifier, on the Debug menu, click Start with Application Verifier
- App Verifier can take you to the line of source code that is causing the fault
- Can detect subtle heap corruption issues
- We use AV extensively to find these kind of problems

App Verifier
- Catch critical errors at run time
- Handle/heap verification issues
- "Breaks" into the line of code in Visual Studio Team System

Code Profiler
Why is profiling important?
- Constant hardware updates; can be expensive
- Poor end-user experience;
- Low throughput of transactions

There are two types of profiling:

- Low overhead
- Periodically checks for what's going on in your application (identify hotspots)

- More intrusive, more exact data
- Break the application into its basic blocks
- Insert timing probes (insert x86, insert MSIL)
- Know how long each method takes

Profiling is typically a complex operation
Have to do lots of nitty gritty work with configuration files
Profiling ASP.NET applications has been especially hard
Can use the Performance Wizard to configure profiling session easily
Can also do more advanced stuff, like memory allocation profiling of managed applications
We combined several tools into an integrated solution (managed, native, memory)

Using Visual Web Developer Web Server
- not susceptible to IIS problems
- can only access locally; mitigates security risk of running a Web server on a client PC

Performance Report Summary View
- make most likely stuff obvious
- Garbage collector activity
- Most called functions
- Most individual work
- Functions that take longest amount of time

Functions View
- flat list of functions called with metrics
- can group by module to focus on your own code

Caller/Callee View (sometimes called the "butterfly view")
- Which function called this function?
- Which functions did this function call?

Callstack View
- Shows all of the entry points
- Hierarchical call view of the functions called

Type View
- Number of instances per type
- Which functions created each type and how many; total bytes, % of total bytes

Code Profiler
- Supports doing more with less (reduce hardware needs by making applications more performant)
- Help ensure a great customer experience

Supporting Operations
AVIcode Demo
- Proactive application monitoring
- Collect enough data if a problem happens (specify which perf counters and windows logs to collect)
- Creates a management pack configuration to use in AVIcode Studio to monitor an application in operations

Comments (1)

Skip to main content