64kb limit on the size of MSMQ Multicast Messages

  • Article

When Windows 2003 came out, Microsoft introduced the ability to broadcast messages to any machines that were listening back. All you had to do was send out a message on a particular port and IP address and any client that had set up a Multicast queue with matching port and IP address would get a copy.

Since its introduction, there have been a couple of security vulnerabilities that needed to be removed:

The second of these, MS08-036, was resolved through an undocumented change in functionality. Basically, a limit of 64kb was put on the maximum size of a message that could be broadcast using the Multicast method. Obviously this has caused a few problems for any existing MSMQ Multicast applications that expected to be able to send larger messages.

[[Edited June 30th 2009, July 21st 2009]] 

A hotfix has been developed to resolve this problem. The KnowledgeBase article is still pending publication but the number is 961605. Use this number to request the hotfix from Product Support Services. The new files should be dated May 20th, 2009.

961605 FIX: Multicast messages larger than 64 kilobytes (KB) are not delivered as expected by using Message Queuing 3.0 after security update MS08-036 is installed

A registry change is required:

  1. Open the registry with Regedit
  2. Navigate to HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\RMCAST\Parameters\
  3. Create a DWord called MaxpacketSize
  4. Set the value to the desired number of bytes.
    You can set it to a value between zero and 4MB.
    If you specify anything above 4MB, it will default to 64K.
  5. A reboot is needed after adding this value.

One workaround would be to roll back the security update but then that would leave the machines vulnerable to a DOS attack so not something I would like to recommend on Internet-facing installations.

Another workaround is to modify the application so that messages are no longer above 64kb - anything larger would require chunking before transmission and rebuilding on receipt.

For some customers neither of these are going to be viable solutions and I would like to hear from those people. If this change in functionality has impacted you, could you please let me know so Microsoft can get an idea of the number, type and size of companies affected?