Hi,
We are migration from NServiceBus 5.2 to 7.2.
Current implementation with V5.
Transport - MSMQ
Persistence - NHibernatePersistence
Logical endpoint count 13 deployed in Azure VMs.
Recently we faced a challenge in the production queue is overloaded by the influx of messages coming. We look to improve concurrency to set each endpoint MaximumConcurrencyLevel value based on Machine(AzureVM core count * 2). Does this core count based concurrency value set will help considering the current endpoint count(13)? Asking this question upfront because currently, we are not using NserviceBus monitoring tools(Service Pulse/Control) and monitoring concurrency is quite sophisticated…
++
What will be the right recommendation path to migrate(v5 to V7) considering the current scale-out scenario. I see Azure Service Bus transport could provide better Scale-out capability and monitoring.
Migrate to V7 with Azure service Bus Vs migrate to V7 with current MSMQ and Sender side distribution.?
I would make small steps at a time and not yet upgrade to Azure Service Bus. You want to be in control of the risk vectors of the upgrade and therefore sticking to the transport you are already familiar with makes most sense to me as a first step.
So I’d say upgrade all endpoints first to v6 using the provided upgrade and migration path
Once you have gone through the migration scenario you have MSMQ running on V6 using Sender Side Distribution. one of the benefits of v6 and MSMQ is also that the MSMQ transport is out of the box quite a bit faster. Once you are there you can upgrade to v7 latest version
After that you might even be able to decommission a few of your machines depending on what your load tests show you.
I would then treat the discussion whether to migrate the transport from MSMQ to Azure ServiceBus a separate thing that can be dealt with once you are running stable and smoothly on v7 with MSMQ and Sender Side Distribution.
Thanks Daniel…!! Related to migration we had discussion on this before with @ramonsmits. I think we are same page.
Current overflown message queue is quite painful now. Could you please put some light on this. Does the approach mentioned in first post of this thread i.e making all 13 endpoint with MaxConcurrencyLevel = corecount * 2 Will help us to gain better throughput…?Or we could may encounter some unforeseen troubles later…?
It can definitely help to increase the throughput.
Or we could may encounter some unforeseen troubles later…?
that is tough to say from the outside. It depends on what the endpoints are doing. Let me make a simple example. Let’s say the handler opens up DBConnections and you have set the DBConnection pool size to 20. In this case, you would probably want to make sure that your concurrency is set to max 20 to not run into connection problems with your Database. So, in essence, I would set the concurrency in alignment with the “weakest” IO element that you are calling from within the handler. Then things should be fine.
I’ll restate earlier recommendations for reference.
Implement server monitoring and monitor CPU, RAM, disk and especially network IO.
Upgrade to version 6 first as this has massive performance improvements and validate if you even need the distributor.
If the system is slow and server monitoring is not indicating bottlenecks, increase maximum concurrency in small steps
If the bottleneck is the machine hosting the endpoints (RAM, CPU, Disk IO), then move endpoints to their own machine.
If you reach a point that all your machines are hosting only a single endpoint than you need to consider sender side distribution if you cannot scale up that specific machine.