A checkin policy to detect that the build is broken in a CI environment

One important aspect of Continuous Integration is getting notification that the build is broken.  One common way is via a desktop notification from an application like TfsAlert that listens for events (or polls if there isn’t a TFS-style event notification mechanism).

Ideally, the build or unit test break would be noticed as soon as it happens, so that the problem can be fixed before other checkins pile up.  Clark Sell has come up with a checkin policy that detects that the last build failed and produces a policy failure when that happens.  That makes it pretty hard to say, “But I didn’t know about the break.”

Stop, the build is broken!!

It’s late, your tired and yet somehow you got stuck fixing the build because junior engineer bob just fired off a check-in and took off.  It looks like the build is now broken because junior hasn’t done a get latest in a month.  In the meantime 10 other people check-in and just end up compounding the problem.

Unfortunately things like this happen.  One day I just got so frustrated, I just wrote a custom TFS check-in policy which gets the last build status and validates a particular build type was actually passing before you check-in.


I’m hoping Clark will post an attachment with the solution in a zip file.

The checkin policy has to make a web service call to find out the state of the last build.  The performance hit of multiple evaluation calls could be reduced by putting in a time window check or doing it asynchronously (checkin policies are expected to be really fast).  However, if you combine this checkin policy with his TfsAlert project and eliminate the call to the build service by changing the checkin policy to use the last build notification from TfsAlert (assuming TfsAlert has gotten at least one build notification), you get double coverage with better performance.

I think this CI checkin policy is a pretty cool idea.

[Update 10/30/2006]  Clark has posted a zip file containing the code in a VS solution, including a setup project.

[Update 3/29/2007]  We’ve added this build checkin policy in Orcas!

tags: , , , ,

Comments (3)

  1. Clark Sell says:

    Since my post yesterday I just decided to update the code and also include Buck’s comments. I even added

  2. don says:

    I haven’t done anything with check-in policies yet but I’m looking to do several things which is when I came across your post.

    I hope this isn’t a stupid question, but how would you check in the fix to the broken build if the policy stops you from checking in? Or does it just give you a warning when you check in?

  3. buckh says:

    Don, the policy will give you a message stating that it has been violated.  You can then choose to override the policy in order to check in anyway (some folks have requested a feature to able to turn off the override, but clearly here that wouldn’t be good).  Presumably, the person doing that is the one that broke the build.