MSMQ messages using HTTP just won’t get delivered #10


This post refers to the situation where a client is sending MSMQ messages over HTTP to a receiving server hidden behind a firewall.

The messages will be stuck in the Outgoing Queue with a state that switches between “Inactive” and “Waiting to Connect.”
If you collect a Network Monitor trace then you should see that the server replied to the message from the client with a response of “HTTP 400 – Bad Request” without including a body in the response.
If you had a PSS engineer format the MSMQLOG.BIN file from the target server then you would see:

[0]694.f28 04/06/2004-15:16:19.608 [lib DestinationQueueToProps] ERROR:Packet is not accepted by QM
[0]694.f28 04/06/2004-15:17:23.636 [qm AppIsDestinationAccepted] ERROR:Packet rejectet because http routing is not supported

The problem here is that the message is being sent to the public IP address bound to the firewall.
The destination server, though, is using a different IP address through Network Address Translation.

As a result, a message being sent to “DIRECT=http://22.33.44.55/msmq/private$/DestinationQueue” reaches the server successfully and IIS delivers the message to MSMQ. However, MSMQ looks at the address and notices that the server does not have a matching IP address assigned to it, having a 192.168.x.y local network address instead. MSMQ decided that the message must be meant for a different machine but there is no routing information available so request is rejected.

The solution is to create a mapping file on the receiving server so that MSMQ knows what to do with messages sent to the 22.33.44.55 address.


The XML mapping file should be located at <%SystemFolder%>\System32\msmq\mappings\ with any filename and looks something like this:

<redirections xmlns=”msmq-queue-redirections.xml”>
    <redirection>
        <from>http://22.33.44.55/msmq/private$/DestinationQueue</from>
        <to>http://192.168.44.55/msmq/private$/RealQueue</to> 
    </redirection>
</redirections>



Notes


  1. The MSMQ service must be restarted for the mapping file to take affect.

  2. The syntax of the routing file is different between Windows Server 2003 and Windows XP.

  3. You can also get this problem when sending messages using a network name that resolves through DNS to the correct IP address of the destination but the machine actually has a different network name.


  4. In general, to test a connection, browse with Internet Explorer to http://receivingMSMQserver/msmq. You should see something like “Error 501/505 – Not implemented or not supported”. This means that you connected successfully but MSMQ was expecting a message and, not surprisingly, complained.

References


Comments (19)

  1. Damian Powell says:

    Hi John,

    That makes senseI have a related problem. My messages get from the client to the server okay, and they are successfully processed. However, the server has a problem sending the ACK (or NACK in the case of a failure) back to the client because the client is itself behind a firewall on another network.

    How on Earth does one work around that?

    If you can’t already tell, I’m an MSMQ noob so be gentle! In our setup, we want to have XP SP2/3 clients communicating with 2003/2003 R2 servers, preferably using WCF.

    Cheers,

    Damian.

  2. MSDNArchive says:

    Hi Damian, the solution is similar to above. You create a "stream receipt" mapping file on the sender (not the destination) as documented in the References link above.

    The file maps a <LogicalAddress> URL of the destination to a <StreamReceiptURL> URL for the corresponding order acknowledgement queue back on the sender. This allows the destination machine to create the correct outgoing queue for the order acks.

    You may then need a redirections mapping file to get the order ack onto the machine if the external and internal names/addresses are different.

    Don’t worry about being an MSMQ noob – it took me 2 hours of testing a few weeks ago to work out that the stream receipt mapping file went on the sending machine and I’m supposed to know what I’m doing!

  3. MSDNArchive says:

    Hi Mark,

    The only thing that looks strange is "xbmlns" in the first line of the mapping file.

    Cheers

    John

  4. Mark says:

    Thanks for noticing that.

    Problem still persists.

    Do I need to set something in IIS as well?

    Also tried, sending to a DestinationQueue like you do, then mapping that to another Queue, I’m assuming your DestinationQueue and RealQueue are on the same machine?

  5. MSDNArchive says:

    Hi Mark

    "Do I need to set something in IIS as well?"

    No, just install MSMQ HTTP support.

    "Also tried, sending to a DestinationQueue like you do, then mapping that to another Queue, I’m assuming your DestinationQueue and RealQueue are on the same machine?"

    Yes and no – the queues don’t have to be. You could, for example, have an Internet-facing MSMQ/web server routing messages to a number of internal MSMQ/web servers based on incoming machine name, IP address or queue name.

    Starting with basics – no mapping file – can you send MSMQ/HTTP messages to the server using its real IP address (http://192.168.254.242/msmq/private$/test)? If that doesn’t succeed then there is no point trying to get the XML mapping file to work. Should the basics fail then you need to look in network traces and IIS logs to see if you can work out the problem. The many other posts I have on this subject (http://blogs.msdn.com/johnbreakwell/archive/tags/MSMQ+over+HTTP/default.aspx) should help here.

    If you can send to http://192.168.254.242/msmq/private$/test then we have a firm base to work on. How do messages addressed to 12.45.170.228 get delivered to the web server? Is this an additional IP address for the machine or are the messages forwarded by a firewall to 192.168.254.242 via publishing rules?

    Cheers

    John Breakwell

  6. Mark says:

    real quick,

    I tried like you stated in your final points,

    http://192.168.254.242/msmq”>http://192.168.254.242/msmq

    and only received a page not found.

    I’m guessing the issue is then with how MSMQ is setup.

    I can get to this local machine’s IIS through

    http://192.168.254.242/

    just fine.

    the MSMQ service is running, I didn’t see anywhere in the properties where you can turn on or off the http functionality only to turn it on hardened MSMQ mode which allows only HTTP/HTTPS messages to be delivered.

    Also says I installed MSMQ in workgroup mode should this be installed in another domain name?

  7. MSDNArchive says:

    Hi Mark,

    I’ve removed the earlier comment as requested.

    If you follow Note 4 above and browse to a website that has MSMQ with HTTP support but don’t get Error 501 then the MSMQ ISAPI application isn’t listening.

    Did you check to see MSMQ with HTTP was installed? (Control Panel, Add or Remove Programs, Windows Components, Application Server, Message Queuing)

    Workgroup mode is fine. No requirement for directory services.

    Cheers

    John Breakwell (MSFT)

  8. Mark says:

    Ok, I now installed HTTP support in the Add/Remove controls portion, now when I do,

    http://192.168.254.242/msmq I’m getting the 501/505 error.

    But still no messages that I’m sending from a client application like I posted above,

    I can’t see what would be wrong on the client side,

    code again:

    MessageQueue Queue = new MessageQueue("FormatName:DIRECT=http://192.168.254.242/msmq/private$/test");

                 System.Messaging.Message Msg2;

                 Msg2 = new System.Messaging.Message();

                 Msg2.Formatter = new ActiveXMessageFormatter();

                 Msg2.Body = textBox1.Text;

                 Queue.Send(Msg2);

  9. Mark says:

    John, thanks for the help so far.

    Still am not seeing the messages on the server side, they’re waiting on my local machine in a "waiting to connect" status.

    Read your other entries involving MSMQ failing to send over HTTP, none of the resolutions worked.

  10. Mark says:

    John, thanks for the feedback, I finally resolved the issue, it was the local Anti-Virus on my machine (the sender), once that was turned off the messages were delivered. 🙂

    -mark

  11. MSDNArchive says:

    Hi Mark,

    Thanks for the update. Did you work out what the AV software was doing? For example, firewalling any HTTP POST requests?

    Cheers

    John Breakwell (MSFT)

  12. David says:

    We are facing something that looks similar;

    the message goes over the internet form A to B.

    It reaches be, and the Outgoing Queue folder shows 0 0 0 , meaning the message reached the destination and it was acknowledged

    on the destination server B, I can see in the IIS logs the post for the msmq/private$/myqueuname 200 …

    but I do not see the message getting to the queue

    this is behind an NLB so the NLB name does not match the computer name … the strange thing is why the source server that sent the message, server A, thinks that it got the message delivered successfuly?

    on the destination server, I see immediately, in the Outgoing Queues, the same address that the source used to send to … the one with the NLB server name; can I assume that the specific computer name behind the NLB thinks the message is not for it and it tries to forward it further … the question is where?

  13. MSDNArchive says:

    Hi David,

    So you are sending transactional message.

    Just because a message does not appear in the queue doesn’t mean it wasn’t delivered. The receiving machine can reject a message for a number of reasons – rejected messages will not go back into the outgoing queuee.

    Instead you want to enable Negative Source Journaling and check the Dead Letter Queue.

    If the machine name is different to that on the message – a common situation for Internet facing machines – then you would set up a mapping file (similar to the one in the blog post above) so that the web server routes the messages to the local machine. If MSMQ has no mapping file, or the mapping file does not contain the machine name on the message, then it will discard the message as it has no knowledge of the destination.

    You may need to set IgnoreOSNameValidation too.

    Cheers

    John Breakwell (MSFT)

  14. Chuck says:

    Hey John – Great blog… I have an interesting scenario. I am running Windows 2003 on Amazon EC2.

    The goal is to use SSL for sending message to MSMQ but… I am trying to just get basic HTTP communication up and going. I have installed WireShark on the client and on the server. WireShark reports the same thing on both Client and Server:

    This is the Server Log – 68.37.172.115 is my clients IP

    17 7.027001 68.37.172.115 10.242.194.150 HTTP POST /msmq/private%24/master HTTP/1.1

    19 7.027641 10.242.194.150 68.37.172.115 HTTP HTTP/1.1 404 Not Found  (text/html)

    Of course being on EC2, the Server IP 10.242.194.150 is not available publicly. I have an ElasticIP for the server that is mapped to a DNS name

    So I set up a map file in system32msmqmapping

    <redirection>

         <from>http://<domain-name>/msmq/private$/master</from&gt;

        <to>http://localhost MSMQ/private$/master</to>

     </redirection>

    If been futzing with this for 2 1/2 days and still cant get a message to MSMQ.

    I have tried various addresses in the <to> tag e.g. localhost,10.242.194.150,127.0.0.1 and even the Elastic IP (I knew it wouldn’t work but – what the hell)

    My client is a windows xp box. Its Outgoing Queue shows no messages waiting and the Queue’s status is Inactive.

    Another thing I am confused over is that there are no entries in my iis logs.

    I’m really stuck and could use a push in the right direction – thanks

  15. MSDNArchive says:

    Hi Chuck,

    This doesn’t look good:

    "HTTP POST /msmq/private%24/master HTTP/1.1"

    The dollar sign in ‘private$’ has been replaced with %24 by your sending application. 0x24 is the ASCII code for the dollar sign.

    No entries in the IIS log is probably an unrelated problem.

    Cheers

    John Breakwell (MSFT)

  16. Chuck says:

    Thanks for the quick response… I tried connecting to the queue with IE and observed WireShark on the Server. The request came over, in IE, as expected…with the private$ as opposed to private%24 – frustrating.

    I changed the <redircetion> entries to read

    <redirection>

         <from>http://<fqdn>/msmq/private%24/Master</from&gt;

         <to>http://localhost/MSMQ/private$/master</to>

     </redirection>  

    But no luck…still with the HTTP 404 error…

  17. MSDNArchive says:

    Hi Chuck,

    I’ve learnt something new – the dollar gets sent over as %24 for the MSMQ POST request in my network trace too but stays as a $ for Internet Explorer’s GET request.

    If you use your Internet browser to navigate to http://<fqdn>/msmq/private$/Master (as described in the Notes section above), what error do you get? Should be "Error 501/505 – Not implemented or not supported". If you get a "HTTP Error 404 – File or directory not found" then the MSMQ virtual directory doesn’t exist (or can’t be reached, maybe). Does MSMQ show as an installed application under the Default Web Site in Computer Management?

    Cheers

    John Breakwell (MSFT)

  18. Chuck says:

    Hey John,

    You need to write a book or get the Nobel MSMQ Prize or something. Anyway, If I navigate, in IE, to

    http://<domain-name>/msmq I do get "Error 501/505"

    if I navigate to http://<domain-name>/msmq/private$/master I get the 404 not found.

    MSMQ is set up as a virtual directory under the default web site.

    I’ve checked and re-checked the settings and locations of the MSMQ virtual directory. I have submitted a support ticket to Amazon EC2, oddly enough, the responded to me with a link to this thread. It maybe time to open a support ticket with MSFT – ouch.

  19. MSDNArchive says:

    Hi Chuck,

    On my test machine, I get an "Error 404" with

    http://server/not_here

    but all the following give "Error 501/505":

    http://server/msmq

    http://server/msmq/private$

    http://server/msmq/private$/queuename

    Maybe this is expected if the IP address is different at the back end.

    You provided a sample redirection file in an earlier comment but it is incomplete and has a typo. What does the real file look like? Like this?

    <redirections xmlns="msmq-queue-redirections.xml">

    <redirection>

     <from>http://<fqdn&gt://msmq/private$/Master</from&gt;

     <to>http://localhost/msmq/private$/Master</to>

    </redirection>

    </redirections>

    To enable IIS logging for the MSMQ messages:

    http://blogs.msdn.com/johnbreakwell/archive/2009/10/21/troubleshooting-msmq-over-http-nothing-in-the-web-server-log-files.aspx