Does MSMQ 3.0 need the RMCAST (PGM) protocol driver?

I noticed in the newsgroups that someone wanted to know how they could remove the PGM protocol from their machines to reduce their exposure to attacks over the network.

In Windows 2003 and XP, this protocol is installed when Message Queuing is selected to provide support for multicasting. Unlike in Windows 2008 and Vista, there is no option to deselect this protocol when installing MSMQ.

MSMQ also has a dependency on the RMCAST (PGM) protocol driver so you can't simply remove the protocol.

If you don't require the ability to multicast MSMQ messages then the workaround for Windows 2003 and XP is as follows:

  1. Open up the Registry Editor
    • Navigate to HKLM\System\CurrentControlSet\Services\MSMQ
    • Edit the DependOnService value and CAREFULLY remove the RMCAST entry.
  2. Open up the Control Panel
    • In Network Connections, locate the Local Area Connection
    • In the connection's General properties, uninstall the "Reliable Multicast Protocol"
  3. Reboot the server to clean up the bindings and dependencies 

[[2nd September 2008]]

Note - this modification will not work for a clustered MSMQ resource because of the way the service is created. MSMQ on a standalone server uses the registry for all of it's settings and so editing the dependencies is simple. A clustered MSMQ resource, though, creates the corresponding service from scratch each time it is started and has a number of settings hard-coded - one of the settings is the service dependencies. So every time when the resource is brought offline/online, the service is re-registered with RMCAST in the dependency list.

This means that if RMCAST is uninstalled, the clustered MSMQ will be unable to start as the dependency list cannot be changed to exclude it. A partial workaround is to deselect "Reliable Multicast Protocol" from each network interface rather than uninstall it.

[[Thanks to Ruud Baars and Xin Chen for their invaluable input]]