Why is there a 4MB limit on MSMQ messages?

  • Article

Here's a request I've received from Dhaval Patel

**

"

I would really like to see some blog entries on how to handle messages greater than 4MB. I have heard some colleagues mention 'COM Wrappers' but I don't like the idea. To begin with, most of us were thinking that starting with Vista, the 4MB limitation would go away... it's a shame that this didn't happen! I was really looking forward to putting MSMQ to good use via WF and WCF, but seeing this limitation is still there, it seems folks like me will just need to stick to SOAP for eternity :)
Please feel free to enlighten me with the history of why MSMQ was forced to handle only 4 MB messages?"

OK, Dhavel, in this blog we will dip into the history behind this limitation.

The first place to start is the excellent MSMQ FAQ document:

3.5 Why is there a 4-MB message size limitation?

There are two main reasons:

  • The Message Queuing driver (Mqac.sys) uses memory-mapped files for message storage, and the present implementation does not break a large message into several message fragments. The driver maps the files to kernel memory address space when Message Queuing runtime sends or receives a message. The kernel memory address space is limited to 16 MB, shared by all drivers and fonts. This is one of the main reasons for the message size limitation. The 16-MB kernel address space limitation also means that a cluster node can host up to three Message Queuing resources, assuming that Message Queuing is not running on the node computer. This is because each resource can consume its own 4-MB range from the kernel address space.
  • The Message Queuing code that manages sessions is not designed to deliver large messages and handle session acknowledgements (ACKs) in the middle of message delivery. This can cause sessions to be closed while a large message is still being sent, because an internal session ACK is not received.

A bit more information on this 16 MB area can be found in the "Deploying Message Queuing (MSMQ) 3.0 in a Server Cluster" document:

Q. Are there any limitations for multiple MSMQ resources in a server cluster?

A. Yes. Each instance of MSMQ on a server node maps 4 MB of the system view space when handling message activity. This results in a default limit of three active, working instances of MSMQ in a cluster node. In a server cluster with three MSMQ resources, a node could have four concurrent MSMQ services running — the service running on the local node plus the three services associated with the MSMQ resources. In such a scenario, message activity might be limited, resulting in resource failures. To overcome this, you should increase the size of the system view space memory pool on each node of a server cluster that is running three or more MSMQ resources. We recommend that you do this even if your node is running less than three.

So we're talking about the system view space, the desktop heap for drivers. Although you can increase the size of this space with the SystemViewSize registry value, this will not affect the MSMQ message size in any way. The registry value is documented on the TechNet website:

Group and resource failure problems

  • The Message Queuing resource fails to handle message activity correctly which may result in resource failures

Although there is another registry value that does seem to set the MSMQ message size, you shouldn't use it. This isn't one of those "It's not supported but here it is for you to try anyway" references - I do actually mean "you shouldn't use it". The value description for MaxMessageSize does say:

Caution

  • Do not change the value of this entry. Increasing this value might prevent you from sending messages larger than 4 MB. Decreasing this value might block transports between Message Queuing computers.

So, in summary, the 4MB limit is there for good design reasons which are still valid in the 21st century.

In the next blog I'll go into ways to get around this limitation.

[[Edited 18th November, 2009]]

Here is an important note from the very useful Resource Management in MSMQ Applications article:

Message Size

MSMQ can support messages under 4 MB in size only, this includes Queued Components messages. Any attempt to send a message through the system that is larger than this will raise the insufficient resources error. Be aware that Unicode data takes up twice as much space as non-Unicode data, as two bytes are needed for each character.