Setting up duplicate detection when integrating data between Dynamics CRM and Dynamics ERP

We’ve seen a few questions come in on the topic of duplicate checking, so wanted to make sure all of you had the info on how this feature is supported with Connector for Microsoft Dynamics.

Before you integrate your Microsoft Dynamics ERP and Microsoft Dynamics CRM, you may have duplicate records between the two applications. For example, if you have entered Daniel Brunner as a contact in Microsoft Dynamics CRM and also as a customer in Microsoft Dynamics AX, you have duplicate records for Daniel Brunner.

Microsoft Dynamics CRM allows users to enter duplicate records, with duplicate keys, for Microsoft Dynamics CRM accounts and contacts. This can cause problems when the integration service tries to integrate the records.

If duplicated records are not identified before you integrate Microsoft Dynamics CRM contacts or accounts with your Microsoft Dynamics ERP customers, you will end with four records for the same person between the two applications.

Set up duplicate detection jobs and rules in Microsoft Dynamics CRM for the account and customer entities to prevent these duplicate record issues.

It’s suggested that you use one of the options listed below that are available in Microsoft Dynamics CRM to keep duplicates from entering the system in the first place. See your Microsoft Dynamics CRM documentation for additional details.

  • Set up Microsoft Dynamics CRM duplicate detection rules In Microsoft Dynamics CRM, set up or modify duplicate detection rules for the entities to integrate. Click Settings > Data Management > Duplicate Detection Rules. The default duplicate detection rules are listed. Use the duplicate detection rules for accounts or contacts using attributes and criteria that make sense for your business rules. Duplicate detection rules in Microsoft Dynamics CRM are based on contact e-mail addresses by default, and the Connector for Microsoft Dynamics maps e-mail addresses based on accounts or customers.
  • Set up Microsoft Dynamics CRM duplicate detection jobs In Microsoft Dynamics CRM, set up duplicate detection jobs for the entities to integrate. Click Settings > Data Management > Duplicate Detection Jobs. Add a job to catch duplicates and notify someone in your organization for each entity to integrate.
Comments (9)

  1. Hi Char, could you just clarify:

    I do initial integration for client who has 12000 Customers in NAV and 20000 Accounts in CRM.

    I know for sure that 12000 Accounts are duplicates for the NAV Customers, and each Account in CRM has NAV Customer No., so defining duplicates is not an issue.

    My question is, if I follow instructions above:

    1. Create Duplication Rules

    2. Create Duplication Detection Job

    3. Run Initial sync

    What I will have after sync will complete?

    A: Is Connector just update duplicate Accounts with Integration Key?

    B: or in CRM appear 12000 additional accounts which recognized as duplicates?

    If (B:), is there any fast way to merge them? Because if merge them one by one, even if spend only 30 seconds per pair, it takes 100 hours …  



  2. thehetz says:

    @NavDeveloper, Connector fully supports CRM duplicate detection rules and messages.  This means that in the scenario that you outline below, when a duplicate record is detected on your initial sync into CRM, CRM will throw a "duplicate detected" exception when the Connector attempts to create a new account.  The CRM adapter will catch this exception, and then send the record back into CRM as an update – rather than a create.  So the answer to your question is A.  After the initial sync, using duplicate detection, you will still have 20000 accounts in CRM and the 12000 accounts that are also in NAV will have been updated to have the same information stored on them that they have in NAV as well as have an integration key assiged to them.

  3. Awsome!! Now it's completely clear.


  4. Hi I am one the persons who asked this question ๐Ÿ™‚ But another issue has arised.

    We have installed the Connector for Microsoft Dynamics and are integrating AX to CRM 2011.

    The issue we have is that we need to do a resync of all the accounts from AX to CRM to get rid of duplicates with the CRM duplicate detection enabled.

    We have tried to delete one of the duplicate accounts created by the connector in CRM and have reset the DAXINTEGRATIONID on the account in AX.

    By this it will "merge" into the existing account in CRM 2011 but it doesn't update the address info.

    We can also see that it doesn't update the DAXINTEGRATIONID column in AX which we think is why it can't find the right addresses to update.

    What do we have to do to be able to resync the accounts from AX to CRM ?

  5. thehetz says:

    @Jack DS – I would strongly recomend that you contact support with this issue as they have worked though it several times for other customers and *might* have scripts that you can use already.  That being said, The DaxIntegrationId from AX is mapped to the KeyId field in CRM, meaning that anywhere in the CRM database that account is refered to (such as on addresses or any lookup to accounts) that GUID will need to be updated to reflect the new value in AX.

  6. I have created a support case with the AX team and hope they can help me out.

    The daxintegrationid on the AX account should be updated with the guid of the merged/updated account in CRM at first. That doesn't seem to happen and that's why I think the address updates doesn't work.

    If anyone else have a clue on this one let me know. If I get a solution from MS support i'll post the answer here if anyone else sometime should need to do a reinitialization.

  7. thehetz says:

    Actually, I think that the unique id of the CRM account should be updated with the DaxIntegrationId from AX first.  But we'll see what support can do ๐Ÿ™‚

  8. thehetz says:

    Actually, I think that the unique id of the CRM account should be updated with the DaxIntegrationId from AX first.  But we'll see what support can do ๐Ÿ™‚

  9. They say that the AX Setupclass should be run again to create new guids in the DAXINTEGRATIONID field.

    I think you are kinda right, the DAXINTEGRATIONID is used to give the new account in CRM a GUID.

    I just think that how should it be able to update the GUID on an existing account by using the duplicate check. Isn't there references many places to that id ? Is an update up the GUID possible via the SDK and does it cascade the change everywhere ??

    We havn't rerun the setupclass yet but have tried to set a new guid manually in the DAXINTEGRATIONID field in AX. It just gives this error:

    VERBOSE TID:11 [2011-09-29T11:02:34.7507049+02:00]: [AX Customer to Account] with MapId [fd330cce-3aec-42ce-b78e-a9ad1aa5841c] is searching for READ keys using 29-09-2011 09:01:49 as last change.

    VERBOSE TID:13 [2011-09-29T11:02:34.7663301+02:00]: [AX Customer to Account] is searching for DELETE keys using 29-09-2011 09:01:49 as last change.

    VERBOSE TID:13 [2011-09-29T11:02:34.7663301+02:00]: [AX Customer to Account] returned no records to delete.

    WARNING TID:11 [2011-09-29T11:02:56.5947345+02:00]: [AX Customer to Account] has encountered an error while processing key [15045].


     Account With Id = ce8d38d9-353f-db11-bcaf-0010c678175d Does Not Exist


                                — Exception Dump —

                                Caught Exception:


     Account With Id = ce8d38d9-353f-db11-bcaf-0010c678175d Does Not Exist


                                Stack trace:

      at Microsoft.Dynamics.Integration.Adapters.Crm40.CrmObjectProvider.CallCrmExecuteWebMethod(Request request)

      at Microsoft.Dynamics.Integration.Adapters.Crm40.CrmObjectProvider.CreateNewDynamicEntity(DynamicEntity entity)

      at Microsoft.Dynamics.Integration.Adapters.Crm40.AccountObjectProvider.CreateUpdateCustomerAddress(Key parentKey, DynamicEntity childEntity)

      at Microsoft.Dynamics.Integration.Adapters.Crm40.AccountObjectProvider.CreateUpdateChildEntities(Dictionary`2 dictionary, Key parentKey, FieldDefinition collectionField)

      at System.Collections.Generic.List`1.ForEach(Action`1 action)

      at Microsoft.Dynamics.Integration.Adapters.Crm40.CrmObjectProvider.WriteParentEntity(Object value, String queryProperty, ICriterion criterion)

      at Microsoft.Dynamics.Integration.Adapters.Crm40.AccountObjectProvider.WriteObject(Object value)

      at Microsoft.Dynamics.Integration.AdapterAbstractionLayer.MixedObjectProviderProxy.WriteObject(Object value)

      at Microsoft.Dynamics.Integration.Service.MapThread.ProcessReads()

                                Inner Exception: Server was unable to process request.

                                Stack trace:

      at System.Web.Services.Protocols.SoapHttpClientProtocol.ReadResponse(SoapClientMessage message, WebResponse response, Stream responseStream, Boolean asyncCall)

      at System.Web.Services.Protocols.SoapHttpClientProtocol.Invoke(String methodName, Object[] parameters)

      at Microsoft.Dynamics.Integration.Adapters.Crm40.CrmWebService.CrmService.Execute(Request Request)

      at Microsoft.Dynamics.Integration.Adapters.Crm40.CrmObjectProvider.CallCrmExecuteWebMethod(Request request)

    INFO TID:10 [2011-09-29T11:02:56.8759881+02:00]: [AX Customer to Account] has completed. 0 record(s) have been written. 0 record(s) have been deleted. 1 record(s) have failed. Total runtime was 22.0940328 seconds.

    It seems to try to update with that guid, but if that is what the AX setupclass does i don't know how to get on with this.

    I'll have a remote session with MS later on today again but the AX team doesn't have that much insight on the connector behaviour ๐Ÿ™

    If anyone here have a clue on the problem i'd appriciate it very much ๐Ÿ™‚

Skip to main content