How to Federate Dynamics CRM

Microsoft CRM Guru Rich Carr has written a great article about how to Federate CRM between systems or tenants.  He is my first “guest blogger” and here is his article.

Federated or Multi-tenant Implementations – How to Sync 1 CRM system to another!

There are some large implementations of CRM that are global enough to consider a federated approach to their implementation. There are also some implementations of XRM where 1 or more entities need to be synced between the tenants in the solution. Then there are a few instances where people may want to go from onPremise to online.

Each one of these scenarios can be solved with variations of the code in this blog.

What started me down this path was curiosity about migrating from onPremise to onLine. There were plenty of whitepapers and documents on going from onLine to onPremise, but nothing going the other way. Research led me to the fact that you have to either Scribe, or try to write some custom .NET code instead.

Given that I love to write .NET code, easy choice. But I certainly didn’t want to hardcode everything!! I started to look at the helper classes and the dynamicentitypartialtype.cs file became the anchor of the solution. If you haven’t tried this helper class out yet, try it – the class is pretty cool. Being a self taught C# coder I spent a little time trying to figure out how it was invoked from the sample code since there was no instantiation of the class. Finally I just ran it and realized that because of how it was constructed that it “just starts” all by itself whenever you instantiate an entity that is a DynamicEntity!

That was cool – but when I tried to add the class to my project with 2 different CrmSdk references, all hell broke loose. The namespace on the class is CrmSdk - since the result of the partial type class was the ability to build a dynamic entity on the fly, and it is a target dynamicentity that I want to create (not source) , I decided to put that class in the same namespace as the target CrmSdk, resulting in:

namespace CrmSdkTarget;

A little more digging to figure out how it really worked and I came across the code where it takes the passed in value and creates an object of the correct type and places the passed in value where it was supposed to go. If you have tried to update or create an entity through the dynamicentity route you have found where an attribute of a certain property has to be set to a value of the correct type. As an example, in order to create a property of type CrmLookupProperty that you have to first create a CrmLookup object and then set the value property of the CrmLookupProperty to this object, like this:

    1: Lookup myLU = new Lookup();
    2: myLU.name = “John Smith”;
    3: myLU.Type = “contact”;
    4: myLU.value = “348145F4-B2F3-DD11-BCF9-00C09F2C80B1”;
    5: LookupProperty myLUProp = new LookupProperty();
    6: myLUProp.Name = “contactid”;
    7: myLUProp.Value = myLU;

Authentication

In my solution I had 2 web references – one called CrmSdkTarget which was for my target – and one called CrmSdkSource which was my source. I called 2 different routines to get an instance of each and inside each routine is the authentication required of each. When I first wrote this I was going from a VPC image on my laptop to my own online system so I needed one AD authentication and one Live. Changing this part of the code should be trivial to match your authentication requirements.

When is DynamicEntity NOT equal to a DynamicEntity?

This is really the crux of the problem. You have an account on system A and want to sync it to system B. In one .NET project you can set up web references to both of the systems and authenticate to them easy enough – but you still can’t grab an account object as a DynamicEntity from System A and set a DynamicEntity from System B equal to it. What’s worse is that when you do anything in your code you have to qualify what you are doing because you now have multiple references to the same object.

In order for this to work I had to add the reference to the Source CrmSdkSource to the dynamicentitypartialtype class – which caused every reference to CrmSdk objects to turn red and ugly because of VS not knowing which one I meant.

So something like:

Public static object GetPropertyValue(Property property)

becomes

Public static object GetPropertyValue(CrmSdkTarget.Property property)

Every single reference to a CRM object had to be typed correctly. Since you can download my code, you don’t have to do it!

Next I tackled the real meat of the solution – the code in the CreateInstance method. This code is where an object is passed and turned into a CRM DynamicEntity Property. Once again – a CrmSdkTarget.Lookup object does NOT equal a CrmSdkSource.Lookup object! So in order for this to work I had to build a CrmSdkTarget.Lookup object by getting the values out of the CrmSdkSource.Lookup object.

This same concept was applied for each CRM property type. The place where things a little more interesting is the DynamicEntityProperty – the property used by activities for things like email to’s and from’s. That can actually be several properties in 1 so the code is a lot messier. Also where it gets confusing is that not all properties on an activityparty are ok for create. For instance while processing the to property for emails the source system will pass you a property within the dynamicentityproperty for donotfax. That is not a valid property for create.

Excluding other properties

In the main driver program there were more issues similar to the to for emails – not every property is ok for create. For instance any of the created and modified properties are read only fields. So I added a switch to exclude these fields. I called the metadata service to get a list of which fields are valid for create and created an ArrayList full of them. The driver program looks at each property in the source and if it is in the ArrayList, it passes it on for creation on the target side.

Finally the code that is used to grab all of the accounts or other entities is pretty much straight from the SDK.

A couple more Gotcha’s

If you are migrating a fully used system then there is a definite order that is required for your migration. For instance Lead has to be done before Account because Accounts have a field called originatingleadid that will throw an error if the lead doesn’t exist yet. There are countless other instances of this. Some of them have to be handled by watching the order – and some a little different.

One that is a little different is the primarycontactid field on account. In my solution I load an account and then save off the primarycontactid field. I then process all of the contacts for the account I just processed and when I find the one that has the same GUID as my saved off primarycontactid, I do an account update.

Another gotch is status and statecode’s. After you create the entity in many cases you have to call a SetState message to get the status to be something other than “New”. The code here accounts for that for all of the entities that are covered.

Finally – activities are just a strange beast no matter how you slice it. A source system will give you a cc or bcc with nothing in it, and creating that same thing isn’t easy. What I decided to do is to check the length of the dynamicentity array and if it is greater than 0 deal with it, otherwise don’t even send it to the target system.

Federation and Multi-Tenancy Timing Considerations

In these scenarios while there are a few options I think I can make a case that there is only one real option. The one option using this code that I would not suggest is to use plug-ins. I don’t suggest this for two reasons – the first is that a web service dynamic entity is not the same as the ICrmService that you get when you do a plug-in. While this is a minor inconvenience and I’m sure could be overcome – the larger issue is one that is more serious.

My background for CRM is in auditing. When you are doing an audit application you cannot “miss” a transaction. As soon as you do that, your application isn’t worth anything. I don’t see anything any different with trying to keep information in sync than in auditing. If you “miss” a transaction, then it is gone and you can’t ever get it back. The one you “miss” could be the one that updates the most important piece of information that someone else really needed to have. Once users lose confidence in your application, then it is a real bear to get it back. How could you “miss” one with a plug-in? If the target system is down for maintenance, in the middle of an IISReset or something similar then your create or update will fail.

So the solution is to ensure delivery by offloading the process. Step 1 is to create a plug-in that just writes out the transaction to a table in a non CRM database. Grab the transaction type, objectId, ObjectTypeCode, preXml, PostXML and timestamp it. If the database write fails, write it all out to a text file. This should never fail to capture the transaction and save it for later processing.

Step 2 is to create a windows service that has a timer. When the timer goes off – first look for text files and then look to the database. Load the text files into the database to ensure first in first out processing.

Step 3 is to create a DLL that the windows service can call that actually does the work – and pass it the transaction. Let that .dll have the multiple web references and be the workhorse. Sprinkle in some heavy duty logging and notifications to folks should things stop working and you have a solution that you know won’t miss any transactions.

The code for this article is posted on MSDN Code Gallery.