How to troubleshoot MSDTC communication failure (II)
After elaborating the check list of MSDTC configuration and setup on Win2003 and Win2008, in this article, we will continue discussing on how to fix below MSDTC communication errors from different parts, including check Firewall and Dynamic Ports allocation:
New transaction cannot enlist in the specified transaction coordinator.
Enlist of MSDTC transaction failed: Result Code = 0x8004d00e
Error 8004d00a. Distributed Transaction error
error 0x8004D025 (XACT_E_PARTNER_NETWORK_TX_DISABLED)
0x8004D01B (The Transaction Manager is not available.)
Section B: Check Firewall settings, which should not block two-directions of MSDTC/RPC communication
1. MSDTC communication fully uses RPC communication, which requires 135 port and a set of dynamic ports, which default is above 1024.
(Windows 2003: 1024~65535; Windows 2008: 49152~65535). Refer to:
832017 Service overview and network port requirements for the Windows Server system
https://support.microsoft.com/default.aspx?scid=kb;EN-US;832017
or You can refer to Section A.5.c to reduce the dynamic port range.
No matter in which situation, make sure the firewall will open two directions communication on those ports for MSDTC client and MSDTC server. If you are using MSDTC Cluster, the physical IP and Virtual cluster IP should also configured as open channel. Refer to
2. If Windows Firewall is turned on, please make sure the related ports are all opened for inbound and outbound, and no RPC restrictions.
On Windows 2008, if the windows firewall is turned on, please make sure to enable all "Distributed Transaction Coordinator" related "Inbound Rules" and "Outbound Rules" in "Windows Firewall with Advanced Security" MMC, please refer to the picture.
3. If the firewall has been configured properly, the DCTPing test tool will provide correct responses.
Download the DTCPing.exe tool, and install this tool on both servers involved. The following file is available for download from the Microsoft Download Center:
Start DTCPing on two machines:
a. You need to replace A and B with proper Network name of your testing machines in below steps.
b. You should only input machine name as is without // or \\
Even though DTCping will work if you give IP address, however the test with IP address is not considered valid for DTC communication.
c. There will be two files for test on each direction.
d. To use this tool, you need to have read/browse right to registry key:
HKEY_CLASSES_ROOT (use the Admin account should be OK):
e. Prepare: copy this exe to both A and B
Test A ---> B:
On Box B, Start dtcping.exe
On Box A, Start dtcping.exe, Type B in the remote host name.
Press Ping Button. If the RPC channel is opened through firewall, we will see " RPC test is successful" without errors.
Close DTCPing tools, start another round test from B --> A
Section C: Check whether RPC dynamic port can be opened without issues
In general scenarios, after go through Section A&B, we can fix most MSDTC communication problems. However, there is one possibility that RPC dynamic port failed to be opened on MSDTC machine and then cause MSDTC communication failure. It is a bit complicated, but we can still leverage some troubleshooting tools to figure it out.
1. Dynamic port range has been used up.
Run "netstat -an>portlist.txt", and check the portlist.txt, to see if the listening port or established port is up to the end dynamic port. If yes, please extend the dynamic port range. Refer to Section A.5.C
2. Dynamic port opening is too slow and MSDTC dropped the connection.
We can verify this situation by capturing Network Trace.
For a good RPC communication, we can see on server side the RPCSS service (running in svchost.exe) can respond end point mapping (ept map) quickly after client requests. In this sample, 202.176 tried to establish RPC communication to 216.245, and 216.245can respond End Point Mapping (ept_map) quickly (in this sample it is 5008):
For a problematic RPC communication, we may see the RPCSS service doesn't respond ept_map at all, there is only ept_map Request, but no any respond from server side:
There can by some different reasons caused this pattern, as far as I know, before RPCSS service gets dynamic RPC port, it will use LSASS to communicate with DC when the server is in domain environment. If the communication channel between LSASS and DC were blocked, then this MSDTC communication will fail. In our sample, the LSASS.exe cannot send data to DC through 49157 port:
To fix this issue, we enable two static ports and open them on the firewall, then MSDTC function becomes normal:
224196 Restricting Active Directory replication traffic and client RPC traffic to a specific port
https://support.microsoft.com/default.aspx?scid=kb;EN-US;224196
Summary
Troubleshooting on MSDTC is more complex than other environments, the check items in Section A & B & C provide most common troubleshooting ideas, which help us resolved many real DTC problems. For some specific MSDTC failure, especially for intermittent failed scenarios, we may need debugging or some trace data for analysis to move forward. But before starting that part, the above steps are always worth checking first.
Thanks!
Freist from Microsoft APGC DSI Team