TFS Integration Platform … To Err is Human … getting the conversion result right: Lesson #1

  • Article

Clipart Illustration of a Full Classroom Of Students Sitting In Their School Desks And Listening To Their Professor As He Stands At A ChalkboardWhen I was debugging the custom adapter we developed for a proof-of-concept tonight, battling to keep my eyes open, I thought of a statement made by Paul Ehrlich … “To err is human, but to really foul up things you need a computer”. When I finally identified the gremlin in the code, I thought that perhaps one should change the statement slightly: “To err is human, but to really foul up things you need a computer programmed by a human”.

NOTE: The following applies to the TFS Integration Platform, in particular the custom adapter development adventure.

Symptom

When running our custom proof-of-concept adapter the synchronization analysis and delta generation worked like a charm. The ProcessChangesGroup method sprung to life and migrated all the changes. So far so good. Except, when the synchronization triggered again, the same changes were processed … again, again and again.

Sharing mistakes made along the way … to ensure they are not repeated!

The log looked the same every time, whereby I am snipping lots from the beginning and the end, to ensure you do not have to scroll through endless verbose logging output :)

[11/5/2009 8:54:34 PM] MigrationConsole.vshost.exe Information: 0 : VersionControl: Processing ChangeGroup #3
[11/5/2009 8:54:34 PM] MigrationConsole.vshost.exe Information: 0 : VersionControl: POC:MP:ProcessChangeGroup - 220
[11/5/2009 8:54:34 PM] MigrationConsole.vshost.exe Information: 0 : VersionControl:      > c:/POCDrop - Microsoft.TeamFoundation.Migration.Tfs2008VCAdapter.TfsMigrationItem
[11/5/2009 8:54:34 PM] MigrationConsole.vshost.exe Information: 0 : VersionControl: Adding folder <c:/POCDrop> to Poc
[11/5/2009 8:54:34 PM] MigrationConsole.vshost.exe Information: 0 : VersionControl: Processing ChangeGroup #4
[11/5/2009 8:54:34 PM] MigrationConsole.vshost.exe Information: 0 : VersionControl: POC:MP:ProcessChangeGroup - 221
[11/5/2009 8:54:34 PM] MigrationConsole.vshost.exe Information: 0 : VersionControl:      > c:/POCDrop/HelloPoc - Microsoft.TeamFoundation.Migration.Tfs2008VCAdapter.TfsMigrationItem
[11/5/2009 8:54:34 PM] MigrationConsole.vshost.exe Information: 0 : VersionControl: Adding folder <c:/POCDrop/HelloPoc> to Poc
[11/5/2009 8:54:34 PM] MigrationConsole.vshost.exe Information: 0 : VersionControl:      > c:/POCDrop/HelloPoc/HelloPoc - Microsoft.TeamFoundation.Migration.Tfs2008VCAdapter.TfsMigrationItem
[11/5/2009 8:54:34 PM] MigrationConsole.vshost.exe Information: 0 : VersionControl: Adding folder <c:/POCDrop/HelloPoc/HelloPoc> to Poc
[11/5/2009 8:54:34 PM] MigrationConsole.vshost.exe Information: 0 : VersionControl:      > c:/POCDrop/HelloPoc/HelloPoc.sln - Microsoft.TeamFoundation.Migration.Tfs2008VCAdapter.TfsMigrationItem
[11/5/2009 8:54:34 PM] MigrationConsole.vshost.exe Information: 0 : VersionControl: Adding file <HelloPoc.sln> to Poc
[11/5/2009 8:54:34 PM] MigrationConsole.vshost.exe Information: 0 : VersionControl:      > c:/POCDrop/HelloPoc/HelloPoc.vssscc - Microsoft.TeamFoundation.Migration.Tfs2008VCAdapter.TfsMigrationItem
[11/5/2009 8:54:34 PM] MigrationConsole.vshost.exe Information: 0 : VersionControl: Adding file <HelloPoc.vssscc> to Poc
[11/5/2009 8:54:34 PM] MigrationConsole.vshost.exe Information: 0 : VersionControl:      > c:/POCDrop/HelloPoc/HelloPoc/HelloPoc.csproj - Microsoft.TeamFoundation.Migration.Tfs2008VCAdapter.TfsMigrationItem
[11/5/2009 8:54:34 PM] MigrationConsole.vshost.exe Information: 0 : VersionControl: Adding file <HelloPoc.csproj> to Poc
[11/5/2009 8:54:34 PM] MigrationConsole.vshost.exe Information: 0 : VersionControl:      > c:/POCDrop/HelloPoc/HelloPoc/HelloPoc.csproj.vspscc - Microsoft.TeamFoundation.Migration.Tfs2008VCAdapter.TfsMigrationItem
[11/5/2009 8:54:34 PM] MigrationConsole.vshost.exe Information: 0 : VersionControl: Adding file <HelloPoc.csproj.vspscc> to Poc
[11/5/2009 8:54:34 PM] MigrationConsole.vshost.exe Information: 0 : VersionControl:      > c:/POCDrop/HelloPoc/HelloPoc/Program.cs - Microsoft.TeamFoundation.Migration.Tfs2008VCAdapter.TfsMigrationItem
[11/5/2009 8:54:34 PM] MigrationConsole.vshost.exe Information: 0 : VersionControl: Adding file <Program.cs> to Poc
[11/5/2009 8:54:34 PM] MigrationConsole.vshost.exe Information: 0 : VersionControl:      > c:/POCDrop/HelloPoc/HelloPoc/Properties - Microsoft.TeamFoundation.Migration.Tfs2008VCAdapter.TfsMigrationItem
[11/5/2009 8:54:34 PM] MigrationConsole.vshost.exe Information: 0 : VersionControl: Adding folder <c:/POCDrop/HelloPoc/HelloPoc/Properties> to Poc
[11/5/2009 8:54:34 PM] MigrationConsole.vshost.exe Information: 0 : VersionControl:      > c:/POCDrop/HelloPoc/HelloPoc/Properties/AssemblyInfo.cs - Microsoft.TeamFoundation.Migration.Tfs2008VCAdapter.TfsMigrationItem
[11/5/2009 8:54:34 PM] MigrationConsole.vshost.exe Information: 0 : VersionControl: Adding file <AssemblyInfo.cs> to Poc

… so why are we creating directories and files over and over again? Don’t we trust the computer to perform the File IO for us?

The Gremlin

The ProcessGroup method looked as follows when I started the investigation:

    1: ConversionResult IMigrationProvider.ProcessChangeGroup(ChangeGroup changeGroup)
    2: {
    3:     ConversionResult convResult = new ConversionResult(Guid.NewGuid(), Guid.NewGuid());
    4:     TraceManager.TraceInformation("POC:MP:ProcessChangeGroup - {0}", changeGroup.Name);
    5:                 
    6:     foreach (MigrationAction action in changeGroup.Actions)
    7:     {
    8:         TraceManager.TraceInformation("     > {0} - {1}", action.Path, action.SourceItem.GetType().ToString());
    9:         if (action.Action == WellKnownChangeActionId.Add && action.ItemTypeReferenceName == WellKnownContentType.VersionControlledFile.ReferenceName)
   10:         {
   11:            // action.SourceItem.Download(action.Path);
   12:             pocUtil.AddFile(action.Path);
   13:         }
   14:         else if (action.Action == WellKnownChangeActionId.Add && action.ItemTypeReferenceName == WellKnownContentType.VersionControlledFolder.ReferenceName)
   15:         {
   16:           //  action.SourceItem.Download(action.Path);
   17:             pocUtil.CreateFolder(action.Path);
   18:         }
   19:     }
   20:     return convResult;
   21: }

Note line 3, which is the root of all evil, as innocent as it may seem. In fact it returned a conversion result to the platform saying “I migrated from who knows, to who cares and the result of the conversion will be left up to your own imagination”. Can we blame the platform for assuming nothing had been done? “No” … in fact the platform should outright punish an adapter that returns such a meaningless conversion result indicator.

Changing the code as follows, allowed me to focus on other work tonight and tick off another gremlin (bug) from our whiteboard. Note that lines 5,6,7, 23 and 24 are different. We identified the source and the migration peer (target), we identified the change identifier for the history.  As the proof-of-concept adapter had no unique change id, such as a TFS changeset number of TFS Work Item Tracking number, we decided to hijack the source ID … not to be repeated in production adapters, but a work around for our proof-of-concept.

    1: ConversionResult IMigrationProvider.ProcessChangeGroup(ChangeGroup changeGroup)
    2: {
    3:     TraceManager.TraceInformation("POC:MP:ProcessChangeGroup - {0}", changeGroup.Name);
    4:     
    5:     Guid targetSideSourceId = m_configurationService.SourceId;
    6:     Guid sourceSideSourceId = m_configurationService.MigrationPeer;
    7:     ConversionResult convResult = new ConversionResult(sourceSideSourceId, targetSideSourceId);
    8:                 
    9:     foreach (MigrationAction action in changeGroup.Actions)
   10:     {
   11:         TraceManager.TraceInformation("     > {0} - {1}", action.Path, action.SourceItem.GetType().ToString());
   12:         if (action.Action == WellKnownChangeActionId.Add && action.ItemTypeReferenceName == WellKnownContentType.VersionControlledFile.ReferenceName)
   13:         {
   14:            // action.SourceItem.Download(action.Path);
   15:             pocUtil.AddFile(action.Path);
   16:         }
   17:         else if (action.Action == WellKnownChangeActionId.Add && action.ItemTypeReferenceName == WellKnownContentType.VersionControlledFolder.ReferenceName)
   18:         {
   19:           //  action.SourceItem.Download(action.Path);
   20:             pocUtil.CreateFolder(action.Path);
   21:         }
   22:     }
   23:     convResult.ChangeId = changeGroup.ReflectedChangeGroupId.ToString(); // for the POC we "loan" the other side's ID
   24:     convResult.ItemConversionHistory.Add(new ItemConversionHistory(changeGroup.Name, string.Empty, convResult.ChangeId, string.Empty));
   25:     return convResult;
   26: }

Lesson #2: According to Terry we have another learning from this adventure. When ConversionResult.ChangeId is left empty, the platform does not mark the MigrationInstruction (nextChangeGroup in the code above) to Complete. As a result, it will be reprocessed in the following round trip.

… so, if you are planning to develop custom adapters for the TFS Integration Platform, make sure you spend time pondering over the high water mark and the conversion result. It could safe you some night debugging, which saves you on burning candles and brewing endless buckets of coffee, which in turn means more sleep and less energy wastage.