BizTalk ENTSSO Service Fails to start (Windows Server 2003)

  • Article

I recently worked on an issue where the customer was not able to start the EntSSO service in their environment. They kept getting below error messages in their event log.

Event Type: Error
Event Source: ENTSSO
Event Category: Enterprise Single Sign-On 
Event ID: 10602
Computer: MACName
Description:
RPC failure. Please check your network configuration. Error Code: 0x80070057, The parameter is incorrect.

Looking at the event log, I also noticed that they were getting below error messages in their event log whenever they start MSDTC service.

Event Type: Error
Event Source: MSDTC
Event Category: CM
Event ID: 4156
Computer: MACName
Description:
String message: RPC raised an exception with a return code RPC_S_INVALIDA_ARG..

Event Type: Warning
Event Source: MSDTC
Event Category: MSDTC Proxy
Event ID: 4358
Computer: ARS-S-10006
Description:
The MS DTC Connection Manager is unable to register with RPC to use TCP/IP or UDP/IP. Please ensure that RPC is configured properly. 
Error Specifics:D:\nt\com\complus\dtc\dtc\cm\src\iomgrsrv.cpp:2220, Pid: 1116
No Callstack, CmdLine: C:\WINDOWS\system32\msdtc.exe

This certainly told me that there is something wrong with either MSDTC or RPC. I consulted the DTC engineer and he told me that this can be due to a corrupt/incorrect registry key under MSDTC hive.He suggested that we uninstall and re-install MSDTC and this should take care of it. We tried the same and it did not work.

We ran DTC Ping on the server and got below results:
When we try to run DTC ping from BizTalk server, we get following error: l_rpcserverallocateipport , parameter is incorrect.

This looked to be a RPC related error. The error talked about allocating Ports for RPC communication. We opened up the registry and went to the hive which is responsible to specify this port allocation.
HKEY_LOCAL_MACHINE\Software\Microsoft\Rpc\Internet\

We noticed two things over here:-       
The key "Ports" was configured as "Port"-       
The value was configured as "1024 to 65535" instead of "1024-65535"

If the Internet hive is not present on the box, you may create the same following the below article :
https://support.microsoft.com/kb/250367/en-us

We changed the values to the expected values and restarted the machine.

Things started working normally once we rebooted the machines. There were no errors in the event log after that and the ENTSSO Service worked just fine.

I hope this helps.

Written By
Yakshit Gohel

Reviewed By
Jainath V R

Microsoft India GTSC