MSMQ, WCF and IIS: Getting them to play nice (Part 1)

A few weeks ago I posted an article describing how my current team built a publish/subscribe message bus using WCF and MSMQ. At that time we had only deployed the application in a single-server test environment. While there were a few tricks to getting that working, once we tried deploying to a multiple server environment we found a whole lot of new challenges. In the end they were all quite solvable, but it seems that not a lot of people have attempted to use the MSMQ bindings for WCF, hosted in IIS 7 WAS, so there isn't a lot of help out on the web. The best source of information I'd found is an MSDN article, Troubleshooting Queued Messaging (which unfortunately I didn't find until after we'd already solved most of our problems). But even that article is a bit lacking, so I thought I'd share some of the things we learned about getting this all working.

The goal here is to set up reliable, asynchronous communication between a client application and a service, which may be on different machines. We will be using MSMQ as a transport mechanism, as it supports reliable queued communication. MSMQ will be deployed on a third server (typically clustered to eliminate a single point of failure). The client application will use WCF's NetMsmqBinding to send messages to a private queue on the MSMQ server. The service will be hosted in IIS 7, and will use Windows Activation Services (WAS) to listen for new messages on the message queue. This listening is done by a Windows Service called SMSvcHost.exe. When a message arrives, it activates the service within an IIS worker process, and the service will process the message. The overall architecture is shown in the following diagram.

Let's start simple by setting everything up on a single server, with no security or transactions to complicate things. This first instalment is a bit of a recap of my earlier post, but I'm including it again here as it will be an important foundation for the more complex steps shown in the next instalments.

Before writing any code, make sure you're running Windows Vista or Windows Server 2008, and that you've installed the following components (I've taken the names from Vista's "Windows Features" dialog; Windows Server 2008 has slightly different options but all should be there somewhere).

1. Microsoft Message Queue (MSMQ) Server > MSMQ Server Core and MSMQ Active Directory Domain Services Integration (needed for Transport Security in Part 2)
2. Microsoft .NET Framework 3.0 > Windows Communication Foundation Non-HTTP Activation
3. Internet Information Services > World Wide Web Services
4. Windows Process Activation Service
5. Distributed Transaction Controller (DTC) - Always installed with Windows Vista, may need to be added for Windows Server 2008

Of course, you'll also want Visual Studio 2005 or 2008 installed so you can write the necessary code.

As with all WCF applications, a great starting point is to define the service contract. The only real trick when building MSMQ services is to ensure that every operation contract is defined with IsOneWay=true. In my example we'll have just one very simple operation, but you could easily add more or use more complicated data contracts.

     [ServiceContract]
    public interface IMsmqContract
    {
        [OperationContract(IsOneWay = true)]
        void SendMessage(string message);
    }

I won't bother with showing any sample client code to call the service, as this is no different from any other WCF client.

Message Queues don't just create themselves, so if you want to build a MSMQ-based application, you'll need to create yourself some queues. The easiest way to do this is from the Computer Management console in Windows Vista, or Server Manager in Windows Server 2008.

In general, message queues can be called whatever you want. However when you are hosting your MSMQ-enabled service in IIS 7 WAS, the queue name must match the URI of your service's .svc file. In this example we'll be hosting the service in an application called MsmqService with an .svc file called MsmqService.svc, so the queue must be called MsmqService/MsmqService.svc. Queues used for WCF services should always be private. While the term "private queue" could imply that the queue cannot be accessed from external machines, this isn't actually true - the only thing that makes a public queue public is that it is published in Active Directory. Since all of our queue paths will be coded into WCF config files, there really isn't any value in publishing the queues to AD.

In this first stage, we won't be using a transactional queue, so make sure you don't click the Transactional checkbox. Transactional queues can add some complexity, but they also provide significantly more reliability so we'll be moving to transactional queues later in the article.

At this time, it's a good idea to configure the security for the queue. You want to make sure that the account running the client is allowed to send messages to the queue, and the account running the service is able to receive messages from the queue. Since the service will be hosted in IIS, by default it will be using the NETWORK SERVICE account.

Now we know the name of the message queue, we can configure the client to send messages to the correct place. First you need to configure a suitable binding. We'll be using the NetMsmqBinding, which is normally the best option when both the client and service are using WCF. For now we will not be using and security or transactions, so we'll need to specify that in the binding (the exactlyOnce="false" attribute means it's non-transactional).

The endpoint definition is defined in the same way as any WCF client endpoint. One thing to look out for is the address syntax for MSMQ services. Rather than using the format name syntax that you may have used in other MSMQ applications, WCF has a new (and simpler) syntax. The key differences are that all slashes go forwards, and you use "private" instead of "private$". So the address for our locally hosted queue will be net.msmq://localhost/private/MsmqService/MsmqService.svc. Here's the complete config file for the client:

 <?xml version="1.0" encoding="utf-8" ?>
<configuration>
  <system.serviceModel>
    <bindings>
      <netMsmqBinding>
        <binding name="MsmqBindingNonTransactionalNoSecurity" exactlyOnce="false">
          <security mode="None"/>
        </binding>
      </netMsmqBinding>
    </bindings>
    <client>
      <endpoint name="MsmqService"
                address="net.msmq://localhost/private/MsmqService/MsmqService.svc"
                binding="netMsmqBinding" bindingConfiguration="MsmqBindingNonTransactionalNoSecurity"
                contract="MsmqContract.IMsmqContract" />
    </client>
  </system.serviceModel>
</configuration>

To create the service, start by setting up a new ASP.NET application, hosted in IIS - just as you would for a normal HTTP-based WCF service. This includes creating a .svc file for the service endpoint, and of course a class that implements the service contract. Again, I won't bother showing this code as it's not specific to MSMQ.

You'll also need to modify the service's web.config file to include the configuration details for your WCF service. Not surprisingly, this will look very similar to what we configured on the client.

   <system.serviceModel>
    <bindings>
      <netMsmqBinding>
        <binding name="MsmqBindingNonTransactionalNoSecurity" exactlyOnce="false">
          <security mode="None"/>
        </binding>
      </netMsmqBinding>
    </bindings>
    <services>
      <service name="MsmqService.MsmqService">
        <endpoint address="net.msmq://localhost/private/MsmqService/MsmqService.svc"
                binding="netMsmqBinding" bindingConfiguration="MsmqBindingNonTransactionalNoSecurity"
                contract="MsmqContract.IMsmqContract" />
      </service>
    </services>
  </system.serviceModel>

The last step is to configure IIS 7 to use WAS to listen to the message queue and activate your service when new messages arrive. There are two parts to this: first you need to activate the net.msmq listener for the entire web site, and second you need to enable the protocols needed for your specific application. You can perform both of these steps using either the appcmd.exe tool (located under C:\Windows\System32\Inetsrv) or by editing the C:\Windows\System32\Inetsrv\config\ApplicationHost.config file in a text editor. Let's go with the former, since it's a bit less dangerous.

To enable the net.msmq listener for the entire web site, use the following command. Note that the bindingInformation='localhost' bit is what tells the listener which machine is hosting the queues that it should listen to. This will be important when we want to start listening to remote queues.

appcmd set site "Default Web Site" -+bindings.[protocol='net.msmq',bindingInformation='localhost']

To enable the net.msmq protocol for our specific application, use the following command. Note that you can configure multiple protocols for a single application, should you want it to be activated in more than one way (for example, to allow either MSMQ or HTTP you could say /enabledProtocols:net.msmq,http).

appcmd set app "Default Web Site/MsmqService" /enabledProtocols:net.msmq

If all has gone to plan, you should be able to successfully send messages from the client to the service, and have the service process them correctly. However if you're anything like me, this probably won't work first time. Troubleshooting MSMQ issues can be somewhat of an art form, but I've listed a few techniques that I've found to be helpful to resolve issues.

• Check queue permissions. Make sure that you've correctly set the ACLs on your message queue so that the user accounts running the client and service are able to send and receive respectively.
• Check the dead letter queues. In many circumstances, MSMQ will send messages to the Dead Letter Queue (or Transactional Dead Letter Queue) if it couldn't successfully be delivered for any reason. Often the details on the dead letter message will explain why it ended up there (for example, you tried sending a non-transactional message to a transactional queue). If you're using MSMQ across multiple machines, make sure you check the Dead Letter Queues on all servers, as it could end up in different places depending on what caused the delivery failure.
• Enable Journaling. Sometimes it can be hard to tell whether a message never arrived at all, or if it arrived and subsequently got "lost". If you enable the "journal" feature on a message queue, you'll see a record of every message that passed through. However use this feature sparingly, as you can very easily end up with a huge number of journal messages after a few hours of testing.
• Shut down the service listener. When troubleshooting, it can be useful to focus on just the client or just the service. For example, if you aren't sure if the client is sending messages properly, you may want to completely disable the service so you can see if the client's messages are arriving on the queue. To do this, you can shut down the IIS service or application pool, or shut down the Net.Msmq Listener Adapter Service.
• Make sure the MSMQ storage isn't maxed out. MSMQ is designed to be resistant against all sorts of failures, such as temporary network outages. However it seems that if you reach the limit of MSMQ's allocated storage, messages will not be delivered at all. This has happened to us a few times after large amounts of messages ended up in the Dead Letter Queue, or when journaling has been left on for too long. It's easy enough to increase the storage limit, but normally when you reach the limit during testing the best thing to do is purge all of your queues.
• Try pinging the service using the browser. When you are working on WCF services exposed through HTTP, you're probably used to hitting the .svc file in a web browser to check that you can receive the metadata correctly and that there are no configuration problems. Unfortunately there isn't any equivalent way to "browse" to an MSMQ service, so simple configuration errors can be very hard to track down. However if you enable the HTTP protocol for your site, you will be able to hit the .svc file in the browser, even if you haven't configured an HTTP endpoint for your service. If you get the standard WCF service page, that means the service is probably configured correctly.

That's it for the basics. In Part 2 of this article, we'll look at what's required to get this application deployed on multiple servers, and specifically focus on what you'll need to do for the security configuration.

• Anonymous
  July 12, 2008
  PingBack from http://www.alvinashcraft.com/2008/07/12/dew-drop-july-12-2008/

• Anonymous
  July 12, 2008
  .NET/C#/Functional Programming The very useful CR_Documentor 2.0 has been released with Sandcastle Preview

• Anonymous
  July 17, 2008
  Previously, in MSMQ, WCF and IIS: Getting them to play nice: In Part 1 , we built a client and IIS-hosted

• Anonymous
  July 17, 2008
  hi, It better if there was a sample project with this. Regards Raag

• Anonymous
  July 18, 2008
  EnjoyA nice series of articles by Tom Hollander Part 1 Part 2 Part 3   Enjoy ....

• Anonymous
  July 18, 2008
  Raag - I've posted the finished sample code as an attachment to Part 3. Enjoy!

• Anonymous
  July 22, 2008
  whilr trying to run the MsmqService Iam getting the following Error. "The protocol 'net.msmq' is not supported" Please giv me the solution. And tell me the steps i need to do in the sample that u have attached with this Blog. Thanks in Advance, Bala

• Anonymous
  July 22, 2008
  Bala - did you run the appcmd scripts to enable the net.msmq protocol for the web site and your specific application?

• Anonymous
  July 28, 2008
  Tom, I had a problem with the appcmd command: C:Program FilesMicrosoft Visual Studio 9.0VC>appcmd set app "Default Web Site /MsmqService" /enabledProtocols:net.msmq ERROR ( message:Must use exact identifer for APP object with verb SET. ) What am I doing wrong?

• Anonymous
  July 28, 2008
  The comment has been removed

• Anonymous
  July 28, 2008
  I see now that you said, "Unfortunately there isn't any equivalent way to "browse" to an MSMQ service, so simple configuration errors can be very hard to track down. " So in the last reply, that's what I was trying to do. After setting the whole site for net.msmq, I get messages in the MSMQ private queue. However, I don't think they are getting picked up by the service. I can't find a trace. Is it writing a trace somewhere? How can I tell if the service is getting the message?

• Anonymous
  July 28, 2008
  Hi Sam - Make sure you run both appcmd scripts successfully. The site-level one is required to enable the binding, and the app-level one to switch it on for that specific app. If you're getting a "protocol not supported" message it looks like you didn't successfully switch it on for the site. If messages are not being picked up from the queue, then either your service isn't listening at all, or it's trying to listen but it's unable to receive for some reason (eg permissions). If you switch on WCF diagnostics (the normal way, using the <system.serviceModel><diagnostics> section) you may get some clues. If nothing is logged, it means the service may not be listening at all. HTH Tom

• Anonymous
  July 28, 2008
  Hi Tom, First thank you for such work, as hosting in WAS is not so frequent... I have the same effect Sam has. Using sample adapted to fit Part I, my queue is filled, but nobody pick up the messages. So I followed your "troubleshooting" section and activate http to check WCF config. And funny part, when I browse the svc, the website starts running and ... the messages are picked up, trace.log generated and messages disappeared from the queue ! wonderful ! Each time I recycle the queue, or restart the website, I have the same effect: need to browse the svc to start the pumping mechanism. I certainly miss something.... Thanks a lot if you have an idea... Sam, do you notice the same effect ? Laurent

• Anonymous
  August 01, 2008
  Tonight&#39;s slide deck is up here . After the presentation, Steve Andrews was kind enough to come all

• Anonymous
  August 05, 2008
  Laurent - this probably means that the net.msmq listener service doesn't have permission to peek from the queue. The service itself is able to receive the messages, but it can't be woken up by the net.msmq listener. Make sure that the account that the net.msmq service runs under (NETWORK SERVICE by default) has the necessary permissions. Tom

• Anonymous
  August 06, 2008
  Tom, Thanks ! this was a security issue. Although NETWORK SERVICE has full control to the queue, it did'nt work. I had to give full right to the computer account (<computername>$) on the queue in order to work. No time to check why, I just wanted to validate we can host WCF services "natively" in Windows Server 2008, I'm happy with that ! Thanks again, Laurent

• Anonymous
  August 07, 2008
  Thanks for a great article about WCF - MSMQ - IIS7 I had trouble removing the messages automatically from the queue and then I found in the Services application a service called: "NetMsmqActivator" with display name: "Net.Msmq Listener Adapter" and desription: "Receives activation requests over the net.msmq and msmq.formatname protocols and passes them to the Windows Process Activation Service." and this service was "disabled" by default on my machine... I changed the startup type to "automatic" and started it, then I received the messages no problem.

• Anonymous
  August 11, 2008
  The comment has been removed

• Anonymous
  August 12, 2008
  轻松玩转 MSMQ, WCF 和 IIS 7.0 MSMQ, WCF and IIS: Getting them to play nice by Tom Hollander Tom Hollander

• Anonymous
  August 12, 2008
  轻松玩转MSMQ,WCF和IIS7.0 MSMQ,WCFandIIS:Gettingthemtoplaynice byTomHollander

• Anonymous
  September 27, 2008
  Welcomeback!InPart1ofthistale,we'dsuccessfullyconfiguredaWCFclientandanIIS-hostedse...

• Anonymous
  September 27, 2008
  Previously,inMSMQ,WCFandIIS:Gettingthemtoplaynice: InPart1,webuiltaclientand...

• Anonymous
  December 02, 2008
  Tonight&#39;s slide deck is up here . After the presentation, Steve Andrews was kind enough to come all the way out to my house, and help fix some workgroup edition permission issues with TFS on Windows Server 2003, and now all the code is checked in

• Anonymous
  February 03, 2009
  Tonight's slide deck is up here . After the presentation, Steve Andrews was kind enough to come all the way out to my house, and help fix some workgroup edition permission issues with TFS on Windows Server 2003, and now all the code is checked in as well.

• Anonymous
  May 31, 2009
  Windows Communication Foundation (WCF) provides a powerful programming model to develop service-oriented