MSQM Store and Forward change to Azure question

,
1

We have been using MSMQ with NSB for a couple of years. The input into our system, “where the first message comes from” is an existing NON-NSB MSMQ queue. We basically convert anything in one queue, into an NSB message and .Send on the bus. Because MSQM is store and forward, we can this way (the input side only) as In-Memory persistence without an issue.

We are looking at converting to Azure Service Bus transport and I’m wondering if wrapping the .Send to Azure inside my non-NSB msmq transaction and checking the task returned from .Send for exceptions is enough to roll back my MSMQ transaction. I see that some things like data being too large is returned outside of send but in that case I have much more control and insight as to what goes in I don’t care about that issue.

Simplified: Can I recreate the simplicity and the level of recoverability with Azure just by checking the Task returned from .Send(msg) ? Or am I going to have to implement different persistence into my send only endpoint?

2

Hi @robm,

Let me rephrase to see if I got this right.
Today you have a non-NServiceBus endpoint that is using native MSMQ implementation to send to your NServiceBus based endpoint (i.e. native integration scenario). NServiceBus based endpoint fetches messages from the MSMQ queue and processes it.

Does that mean you use in-memory persistence as you are not using any persistence-based features with MSMQ?

Now you’re evolving your system to work with Azure Service Bus transport and would like to replicate the same “safety” mechanism where your non-NServiceBus endpoint could send ASB messages reliably.

We are looking at converting to Azure Service Bus transport and I’m wondering if wrapping the .Send to Azure inside my non-NSB msmq transaction and checking the task returned from .Send for exceptions is enough to roll back my MSMQ transaction.

You could take this approach. You would not need to verify the Send task result. Just invoke the send. If it fails, the message has obviously failed to be sent and you won’t remove MSMQ message.

I see that some things like data being too large is returned outside of send but in that case I have much more control and insight as to what goes in I don’t care about that issue.

Not sure I follow. ASB client/service limits messages in size. If your BrokeredMessage exceeds the maximum size, Send operation will fail. If you need to send messages larger than the limit on the ASB tier you’re using, you can use Azure DataBus feature.

Let me know if that’s what you were after.

Thanks,
Sean

3

Almost…
See if this pseudo code explains it…

ConfigureNSBSendOnlyEndpoint();
var myMSMQConn = System.Messaging.MessageQueue(“inputQueue”);
while( true)
{
myTx = SystemMessagingMessageQueueTransaction.Begin();
myMessage = inputQueue.Recieve( mytrans);
myNSBMsg = CustomParseAndConvertMSMQIntoNSBCommandObject( myMessage);
Bus.Send( myNSBMsg);
myTx.Commit();
}

This works well because MSMQ is store and Forward so the NSB Bus.Send() call is persisted to disk and MSMQ handles it.

This seems problematic with v6 with Task based .Send() and Azure. I’m wondering if it is sufficient to check the returned task from NSB .Send() after it completes and if there are no exceptions in it, then do my myTx.Commit(); ? I use the Persistence features on all the other NSB endpoints, but doing that on this particular SendOnly endpoint is problematic.

4

NServiceBus persistence is required for operations that are not supported by the native underlying transport technology.
For example, Sagas. There’s no concept of a Saga built into ASB. For that, persistence is required. In your case, you’re doing a straight send to ASB, not Persistence is needed IMO.

Could you please elaborate on what’s problematic? If you are still retrieving your original messages from MSMQ and use IMessageSession to dispatch immediately message to ASB, if sending task that is awaited is failing, it means messages was not sent successfully and you would not commit the transaction. Therefore MSMQ message will not be removed.

5

That’s exactly what I was looking for, confirmation that checking that nsb task for exceptions/failing is enough to know the message was sent, and any connection problem to azure would be reflected in that result, giving me clear understanding of when to commit (or not) my native msmq transaction outside of nsb.

So it sounds like this will achieve what I want for my send only endpoint that is running off input from my non msmq pump.

thanks :slight_smile:

edit: I say nsb persistence is problematic because I either have to have an sql server (not practical on my initial send endpoint) or use Raven. We tried Raven and found that the db corruption in Raven that occurs on our endpoint from devices being powered off randomly was a larger problem than the super slim possibility that we would loose a single message with msmq store and forward.