How do I get transactional remote receives with MSMQ?


It is not always efficient for business reasons to implement a classic push system - "send remote, read local" - and a pull mechanism may instead be required - "send local, read remote". The downside to this has been that the "read remote" couild not be transactional which makes the operation unreliable and prone to data loss.


Transactional Remote Receive
"However, there are cases in which a transactional remote receive is necessary. For example, a single message queue receives work orders, and applications on other computers read these work orders and process them. If one of the applications is unable to process a work order, we want the work order to be returned to the queue, rather than dropped."


So, how can you get transactional remote receives with MSMQ?


The options are:



  • Use Windows Vista (and Windows 2008 server when available) as transactional remote receive has been implemented.


Message Queuing features
"New in Message Queuing 4.0, transactional remote receive is a transactional receive of a message from a remote queue. Transactional remote receive capability may simplify or enable certain message processing scenarios. For example, when work orders from a remote central queue need to be processed across a farm of application servers, a transactional remote receive will enable the message processing to be load-balanced across the server farm."



  • Implement a dispatcher application that pushes messages from the server to the receiver so that the application can then perform a transactional local receive. That is:


    • Message arrives in the main queue

    • Dispatcher reads message from main queue within a transaction

    • Dispatcher sends message to a remote processing queue on the client within a transaction

    • Client reads message from local queue within a transaction


This is discussed further here:



Microsoft Message Queuing Services (MSMQ) Tips
"Transactional Remote Read Semantics in MSMQ 1.0"



  • Implement a peek-then-receive process to simulate a transaction.
    The principle is as follows:


Remote Peek
Application (on the client) performs a remote PEEK on the queue (on the server). Message properties (label, body, etc) are read but the message itself remains in place. The application performs whatever operations it needs to with the message data (updates records, etc.) within a local DTC transaction. If there is a failure on the client here then the message will still exist in the queue on the server and processing can be attempted again when the application restarts.


Remote Receive
When these operations are completed, the application performs a remote RECEIVE on the same message which removes it from the queue. The application has already processed the message’s data so the received message is simply discarded. If there is a failure on the client here then duplicate processing may result - the message has not been removed and the application may then try to process it all over again on its next iteration through the messages in the queue. It is therefore necessary to have an extra step in the "Remote Peek" phase that checks if the PEEKed message has already been processed.

Comments (3)

  1. James Dawson says:

    Thanks for the interesting post John – on my current project we have implemented the dispatcher approach.  Am I missing something, or is the ‘peek-then-receive’ model only really viable where you have a single-threaded client process?  (i.e. to avoid processing duplicate messages).

    Thanks,

    James.

  2. MSDN Archive says:

    Hi James,

    No, you are not missing anything. As you have correctly highlighted, the ‘peek-then-receive’ model does not scale.

    I wouldn’t recommend using this approach with multiple clients accessing the same remote queue. You could add extra functionality to prevent the clients processing each other’s messages but the application may become very unwieldy and inefficient.

    Cheers

    John

  3. Johanlbotha says:

    James,

    Yes an No, I used that approach with a pattern that peeks using a cursor and throws messages onto the threadpool for processing. Of course, use a counting semaphore to throttle how many you process at a time. I have no scaling problems at all. However, as John pointed out, this works only on a single reader on the queue, we use dispatching to distribute load balanced (random number based) to the app servers. Anything that uses a single point will only scale so far, and with MSMQ as with SQL you eventually run into network and disk performance issues.

    Yup, I know this post is old, but Johns blog here is one of the best resouces for people using MSMQ, so I leave it here for the next person to find. 🙂

    Johan

Skip to main content