MSDTC is RPC application, so if you are running transaction between two machines, MSDTC will use RPC protocol to communicate with remote machine
RPC is heart of all Microsoft Distributed application (not in web service world),MSDTC,COM,COM+ will use RPC protocol to communicate with remote machine.
It is very easy to understand RPC basics, the way we did in C++, define the interface and server will implement this interface, client will use this Interface to reach remote server, in RPC world, they generate proxy/stub from this interface, and RPC will use proxy/stub to exchange data between client and server.
Any application (console/windows service) can be RPC application, they have to follow certain guide lines, you can know more about RPC here
HOW TO MONITOR MSDTC COMMUNICATION:
(2)You can use Microsoft Network monitor or ethereal(http://www.ethereal.com) to monitor MSDTC traffic between two machines (once you got network trace, filter it by RPC or DCERPC)
(3)to reach remote machine,MSDTC will use NETBIOS to resolve remote machine name (This means you should be able to resolve remote machine name by NEBIOS name(Just resolving remote machine name by fully qualified name is not enough)
(4)so now we know MSDTC is windows service and it will use RPC protocol and it will use NETBIOS to resolve remote machine names.
(5)If you are not able to run Transactions between two machines, first make sure Name resolution, RPC communication is working between two machines
You can ping by short name (for example, my machine name is TESTMACHINE.REDMONDDOMAIN.MICROSOFT.COM), ping by TESTMACHINE(short name)
Name resolution has to work in both directions, on both machines; you should be able to resolve both machine names by NETBIOS name.
In cluster case, you should be able to resolve MSDTC/cluster virtual names from all cluster nodes and client machine.
Names and IP addresses that an MSDTC client in a cluster environment must have
If you have firewall between two machines, most likely they might have restricted number of ports, if you didn't configure these machines for RPC
Communication, RPC applications (MSDTC also RPC app) can choose any port between 1024-65000, these ports may not be in firewall range, so firewall may block RPC communication
What you have to do:
Please choose certain range (let’s say 5000-5020), and configure this range on both machines, and open the same range in firewall, so all RPC application will choose RPC port in this range, since firewall knows about this range, it will allow the communication
Using Distributed COM with Firewalls(DCOM also uses RPC)
How to trouble shoot MSDTC firewall communication
If you have windows firewall (most likely on xp, sp2 and 2003, sp1), open port 135 and add MSDTC to exception list
(6)My team wrote DTCPING tool to trouble shoot name resolution/RPC communication to trouble shoot MSDTC problems
You can download DTCPING from here
This KB article also gives you instructions on how to use DTCPing.
HOW TO RUN DTCPING:
(1) Start DTCPING.exe on both machines (let say on Machine1 and Machine2)
(2) from Machine1, ping Machine2
(3)from Machine2,ping Machine1
Once you are done with test, please close DTCPING.exe on both machines, if you want to test again, close current DTCPING.exe and start dtcping.exe again on both machines
Once you got dtcping log files, look for error message, mostly it will report following problems
(1)Name resolution problem (not able to resolve remote machine name by NEBIOS name)
(2)RPC communication problem (it will show us RPC communication went ok or not)
(3)it will read all ipconfig info and MSDTC security settings on XP,2003 machines
DTCPING is working doesn't mean that MSDTC going to work, DTCPING will test RPC communication, so if DTCPING communication working means, DTC communication also may work, because of MSDTC security, we may still see transaction problem; I will cover more about MSDTC security in next blog.
I hope, I covered enough about MSDTC RPC communication, and How to trouble shoot MSDTC rpc communication problems, if you have any questions on MSDTC rpc communications, please let me know