TFS Integration Tools – I updated the session configuration … why are my newly added work item types not migrating? Q&A-42

Article
01/05/2011

In a recent testing exercise we noticed what we thought was a bug, until we investigated and drew the evidence on a whiteboard. Coffee break … “tada” … not a bug, a misunderstanding. As it is probably a common one, we thought it best to summarise in a quick blog post.

Environment

The environment is a simple migration of work items from ClearQuest to Team Foundation Server, using the TFS Integration Tools. As shown, we are migrating request and task work items from ClearQuest to User Story and Task work item types on TFS respectively … the migration is working like a charm :)

Change in scope … what? Scope creep … no, business requirements alignment :|

Then the unthinkable happens … a change in scope, requested by business, to migrate the defect work items from ClearQuest to Bug work item types on TFS as well.

Quick change to TFS Integration Tools migration session configuration

The change is fairly simple. We add a work item type map from Defect to Bug as shown below, as well as field and value mappings not shown in the illustration.

Where is my stuff … anomaly?

We then trigger another round of migration, are happy to see that we have no conflicts … which means our mapping configuration is correct … but we notice that none of the defects have been migrated to bugs. To make matters worse, we cannot find any errors in the migration logs either.

Huh? … this is where we were before visiting the whiteboard and coffee machine.

Understanding the anomaly

What we realised eventually is that:

The migration process looks for changes using the high water mark (HWM). In case of ClearQuest the HWM is based on the date and time stamp of the work items.
As we have completed the migration before introducing the scope change (adding defect –> bug mapping) the HWM is already set to the highest date/time value of the migrated data (Request and Task work items in this case). Any work items, of type Defect, which have a date less or equal to the watermark will therefore be ignored.

To resolve this misunderstanding or anomaly you have to create a new configuration session and re-run the migration. It is very important to note that by creating a new session with all three work item types we will migrate the already migrated work items (request and task) again, resulting in duplicate WIT items on TFS.

This highlights the importance of testing your migration configuration in isolated test environments, migrating production data only once you have tested and finalized the configuration. As recommended in the readiness kit that ships with the TFS Integration Tools, plan^5, test^18, then migrate.