Connecting to CRM Online 2011 Web Services using SOAP Requests Only


Updated 29th August 2012: The sample has been modified to support both the Microsoft online services environment (Office 365 style) authentication and the Microsoft account (formerly Windows Live ID) authentication. Once again many thanks to Ryan Lo from Marketo for providing the updated sample.

Recently I have had a number of developers asking me about using CRM Online web services from within non.NET applications such as PHP or Flash web apps. Given my heads down approach on .NET these days I ended up putting together a pretty basic sample in C# that shows how to access CRM Online web services using SOAP messages only. This code sample retrieves data from CRM Online using pure SOAP calls only and no additional assemblies to illustrate the underlying SOAP interactions. Couple of ISV partners were able to successfully use this sample and port it over to PHP & Adobe ActionScript, so this stuff apparently works!

The soap messages were based on Fiddler (http://www.fiddler2.com) traffic capture of sample code from the CRM 2011 SDK (http://msdn.microsoft.com/en-us/library/gg309408.aspx).

You can download the sample code here:
http://code.msdn.microsoft.com/CRM-Online-2011-WebServices-14913a16

At a high-level below is what the code does:

  1. Pass in the device credentials and get a PUID. The device credentials is a randomly generated string that satisfies Live ID schema. You can generate one from this tool: Create CRM 2011 Beta Device
    1. POST https://login.live.com/ppsecure/DeviceAddCredential.srf
    2. Get the PUID from response
  2. Pass the device credentials
    1. POST https://login.live.com/****STS.srf (dynamically obtained) 
    2. Get the device CiperData (BinaryDAToken)
  3. Pass the WLID username, password and device BinaryDAToken
    1. POST https://login.live.com/***STS.srf (dynamically obtained)
    2. Get the security tokens (2 CipherValues) & X509SubjectKeyIdentifier
  4. Do CRUD with the web service by passing X509SubjectKeyIdentifier, 2 CipherValues and the SOAP request (with data payload)
    1. POST https://yourorganization.api.crm.dynamics.com/XRMServices/2011/Organization.svc
    2. Get the result from the CRUD response and parse XML to get the data you need

 

BTW the new MSDN code gallery (beta) is pretty awesome with the ability to browse through the source code without downloading and extracting the zip file.
Pretty nice, huh!

image

Comments (43)

  1. lukas says:

    First of all, thank you very much for your example. It is solving exactly the issue I need to archive.

    However I'm not able to pass through 4th step. While calling  request.GetResponse(), I always get 'The remote server returned an error: (500) Internal Server Error.' I'm pretty sure my credentials and CRMUrl are correctly set. Do you have any ideas what I could have missed?

    Thanks a lot for any suggestions, Lukas

  2. Ganwell says:

    Hello, I keep getting "An error occurred when verifying security for the message." I get it with the code from your zip and my test. I tried to verify the messages using fiddler and it seems ok. Any idea what I could try?

  3. Girish Raja says:

    @lukas Have your set the correct account ID from CRM Online? Many have had similar issues where they didn't specify the correct account ID causing the code to fail.

    @Ganwell That's a strange message. Is there a proxy or something that you need in your environment? Also if you have https bypass on fiddler and leave it running, it might give you errors as well. Close fiddler and see if it helps.

  4. alya says:

    @Ganwell did you find a solution for your problem? I have the same error message…

  5. Girish Raja says:

    The sample has been modified to support CRM Online accounts in Europe & Asia. For more details regarding the fix, please see: blogs.msdn.com/…/crm-online-2011-web-services-soap-sample-now-ready-for-emea-amp-asia.aspx

    Can you please run the udpated sample and see if you're still getting errors?

    Thanks,

    -Girish.

  6. Steve says:

    Hi Girish,

    Thank you. You mentioned in your post that "Couple of ISV partners were able to successfully use this sample and port it over to PHP…"

    Can you point me in the right direction of an example to see how this was implemented in PHP?

  7. Sriraja says:

    Steve – we have a working example with PHP.

    You can try it out here…

    dynamics.21logs.com/webclient

    email me at sraja ==at== 21logs dottt com, i will send u the source…

  8. James says:

    HI Girish ,

      Sweet….Great example very helpful .great work , works great with apache HttpClient as well…cool

  9. nadrahan says:

    Thanks for such a nice Post. I really enjoyed and learned a lot from it. I will wait for content from your Side. Thanks for sharing such valuable Views.

    http://imraan.com

  10. Prabhu says:

    James Colud please share the code…

    @Ganwell did you find a solution for your problem? I have the same error message…

  11. Melanie says:

    Hi, thanks for providing this code.  I'm looking to integrate our Dynamics 2011 IFD with our company website, which is built in PHP.  We want to be able to give clients the ability to view and update their contact records and upload and download files after logging in.

    I'm aware of the various Customer Portal plugins that exist but as they are built in aspx and our site is in php, I was to explore various other options before going down that path.

    Do you think the procedure you've outlined in this post is secure enough to transfer client details from CRM to our site?  We will be displaying confidential financial information, so this is a real concern for us.

    Thanks in advance for any pointers you can give me.

  12. Girish Raja says:

    @Melanie

    Yes, you should be able to use a similar procedure to securely integrate your PHP website with CRM 2011 IFD setup. As I have done here, you should create a basic .NET sample and capture the network traffic using IE9 or Fiddler. You can then play it back using code on your PHP site.

  13. Adam says:

    I am in a similar situation to @Melanie.  I'm working on a PHP site for a customer that wants to drive its customer data via CRM 2011 On Premise setup for IFD.  The biggest question I need answered at this point is, is it possible to use the OData Service from PHP with IFD?  Prior to switching to IFD, the CRM was setup for simple AD/NTLM authentication.  We had the site almost complete working using the OData Service.  The CRM was switch over to IFD in order to meet other requirements.  Since then, I've been trying to figure out how to make the OData Service code work, but have been unable to make it happen.  

    Can you shed any light on this for me?

  14. Girish Raja says:

    @Adam

    The OData endpoint at the moment doesn't have any external authentication mechanism. If you're using IFD and calling from an external app (PHP in this case), you'd have to use the regular WCF endpoint instead.

  15. Adam says:

    Thanks for that.  I was kinda thinking that I wouldn't be able to use OData in the IFD environment.

    Can you give me any guidance about the URLs to use to authenticate to the On Premise IFD environment?  The code you've given is for the Online version, is there a similar mechanism available in On Premise?

  16. Cheng says:

    I can finally get the result from the CRUD response and parse XML to get the data, accountid and account name. But based on the account info, how can I update the other info of the accounts?

  17. Girish Raja says:

    @Cheng, You'd have to compose the appropriate XML fragment to update the account. Some examples of the update XML are available in the CRM Online SDK.

  18. Cheng says:

    Can you please point me which part of CRM Online SDK has that sample code?

  19. Mark Rohde says:

    Here we are March 2012 and this is still relevant! Thanks for the post it's solved a big headache .

  20. Guru Prasad says:

    How to implement this for Online Federated Authentication? New Online Federated CRM version.

  21. Peter Lai says:

    I'm getting 'The remote server returned an error: (500) Internal Server Error.' on my new development too.  @Lucas have you resolved your issue on this 500 error code yet?  Can you share your resolution?

    @Girish posted "Have your set the correct account ID from CRM Online?" on this issue.   What do you mean?  Where do I do it in CRM Online?   Sorry, but I'm new to this CRM online right now.   Many thanks!

  22. Swapan Rajdev says:

    Girish – this is a great solution, thanks a lot or it.

    It was working perfectly for us till a week ago, but now i get 500 internal server error on step 4 everytime. Anyone experiencing the same problem?

  23. Girish Raja says:

    Peter / Swapan,

    Recently some of the Windows Live ID login URLs & URNs changed from previous values. That caused the code to fail with 500 internal server error. The sample has now been updated to dynamically pull the Windows Live URLs & URNs from the values obtained from discovery service. If you use the updated sample, your issue should be resolved. Hope that helps.

    Thanks,

    -Girish.

  24. Girish Raja says:

    @GuruPrasad,

    Unfortunately I haven't had the chance to rebuild this sample for federated identity yet. But you should be able to capture the network traffic in fiddler using the same approach I mention above to build a sample for federated authentication.

  25. Alex says:

    Does not work. error 500 when sends headers to CRM Online.

    (North Europe)

  26. Girish Raja says:

    Alex,

    Does your CRMOnline instance use Windows Live ID (hotmail) for authentication? If it uses a different authentication method (Office365 ID or Microsoft Online Services Portal) then this sample won't work.

  27. Lourdhu says:

    Hi Girish Raja,

    The latest code failing at step2.

    – used xxxx@xxx.onmicrosoft.com as userid.

    – also observed that XML formation for Step 2 referring parameter 4 instead of 5 for STSEndpoint.

      i.e {5:STSEndpoint}</a:To>

    – The XML response from server is "The entered and stored passwords do not match."

    Any help would be appreciated.

  28. Girish Raja says:

    @Lourdhu

    The code sample has been updated to reflect the new OCP (office 365 style) authentication. I’ll shortly blog about the details of the updated sample but if you re-download the sample, you should be good to go.

  29. AmitD says:

    Had been able to integrate using LiveId.

    Facing problem using office 365…. however authentication was successful using the attached code but CRM communication resulted in : An error occurred when verifying security for the message. Anyone able to integrate this in Java/Open source?

  30. Noozan says:

    Can anyone help me to connect php with crm 2011 office 365 authentication.. please help me.. you can contact with me via noozaan@hotmail.com

  31. yashaswi says:

    Super Girish..

           can u please tell me how to get all contacts in single file without mentioning contact id(i.e., i want all contacts in particular user) . it's urgent…………

  32. Microsoft Dynamics CRM Solutions says:

    Thanks for a great post. It will be very helpful for learning. This view would be very useful. Thanks for sharing this great article.

  33. Ken Heiman says:

    Any chance of someone as talented as yourself working through the reverse engineering for an ADFS/IFD On-Premise environment instead of just online?  🙂  Getting a token from ADFS is not bad, but the XML for an Organization.svc request is a lot more hairy and I need the support of some crypto folks to understand how to reverse engineer it. See social.microsoft.com/…/c485d98b-6e0b-49e7-ab34-8ecf8d694d31 for an example of a 'successful' on-premise IFD request – it has a whole different layout with other things that need to be generated and its not clear to me (at all) how to get there.

  34. Ken Heiman says:

    After learning more than i wanted about crypto, I solved the on-premise case. Thanks again for this great starting point.

  35. Girish Raja says:

    @Ken,

    That is great to hear. If you have any code samples and/or pointers to share with the rest of the community, I'm sure that would be very welcome!

  36. Sagar says:

    Hi Girish,

    You work for us is really awesome, appreciate it.

    I had an issue connecting to CRM 2011 using java.

    The CRM system doesnt use live id for authentication.It uses the active directory.

    I am clue less how to goahead.I am new to CRM.

    Can you please suggest me a way.

    Thanks,

    Sagar

  37. Ratnesh Jain says:

    Is there any way to support OATH Authetication on Microsoft Dynamic.

    Using your above code, we are able to authenticate and able to get List of Accounts. Thanks for Gr8 and Simplified post.

  38. Rodrigo says:

    not work error 500 with crm 2013 on line

  39. Girish Raja says:

    Rodrigo,

    This has not been updated for CRM 2013. You're better off using the OData endpoint of CRM instead of this workaround for accessing CRM Online web services with the 2013 release.

  40. Kyle R says:

    Hi Girish,

    Any plans to update this for CRM 2013 online?

  41. Shaffic says:

    Dear ppl,

    Anybody have connected to CRM 2011 on premises using Active Direrctory from JAVA.

    If so how did you achieve.. i am very much clueless.. any help will be appreciated.

    Thanks

  42. Ana says:

    Shaffic,

    Any progress on porting the solution to Java?

    Thanks,

    Ana

  43. Neha says:

    Guys any update on connection to crm using JAVA??