"What are MSMQ's limits?" If I had a farthing for every time...
To really get into the right mind-set for discussing MSMQ's limits I think we should first read the words of that great man Douglas Adams:
"Space is big - really big - you just won't believe how vastly, hugely mind-bogglingly big it is. You may think it's a long way down the road to the chemist, but that's just peanuts to space."
And so it is with MSMQ. The design of MSMQ, or more accurately the underlying operating system, means it can manage more messages than most companies will send in their lifetime.
But there are other similarities between space and MSMQ. To get into space you need the budget of the National Aeronautics and Space Administration and to max out MSMQ you would have to spend money on the most powerful datacentre server on the planet with hundreds of terabytes of disk space and hundreds of gigabytes of RAM. I am not even sure if the server hardware available today is capable of letting MSMQ achieve what the operating system will let it do. Or if the CIO exists who would sign off the purchase order for it...
And what are you doing trying to store large quantities data in MSMQ? Message queuing is a transport mechanism, not a storage product. If you frequently find yourself using "MSMQ" and "storage" in the same sentence then you are on the wrong track - what would you be doing with a terabyte of messages, for example? Or several million messages? In that case you want SQL Server.
So maybe we should leave it there.
Well, if you insist... I'll give you the guidance in working out what you need:
- Maximum volume of messages
And this applies to whether you are talking about in an individual queue or as a whole system as that's the same thing.
174307 Interpreting file names in the Storage directory informs us that MSMQ uses pairs of files for persisting messages (one of 4mb for the message data and the other of only 8kb for managing the storage file). So the limit is how much free space there is on the volume you have created on the disk in your server - it is therefore best that MSMQ is using a dedicated volume and not sharing it to get the most out of the volume. Simple as that - create a larger volume and increase MSMQ's capacity for retaining messages.
Focus on your design specification - if your system sends 100mb of messages every hour and you want to be able to cope with 24 hours of down time then you only need a few gigabytes of disk space. If you start looking at the price of terabyte storage area networks then chances are your design spec is wrong.
- Maximum number of messages
This one is not as simple to work out. From my Insufficient Resources post we know that each message needs 75 bytes of kernel memory for indexing so, for example, 2 million messsages would require roughly 150 megabytes. It would seem, therefore, that all you need to do is add more RAM. After looking at a comparison of 32-bit and 64-bit memory architectures, though, you will quickly have to move to the 64-bit platform to take advantage of your investment as 32-bit machines max out at 450 MB of paged pool memory regardless of the amount of RAM fitted.
But, again, if you are trying to work out what amount of RAM will generate the paged pool memory required to accommodate a billion MSMQ messages, your design spec is up for some serious reviewing.
- Other things to consider
Assuming you are able to build the machine of your dreams with terabytes of disk space and gigabytes of RAM, how are you going to get to those messages? Such a beefy server is going to be the central hub of a very busy system, probably with many remote users, and that means the throughput of the disk and network are going to be critical - imagine having a warehouse the size of a Boeing 747 aircraft hanger with one fork-lift truck to move the stock around with.
I hope that helps. Yes, I realise I cheated and didn't really give you MSMQ's limits but what would you do with the numbers anyway?