Migration strategy MsMq to RabbitMq

,
#1

We want to migrate from MsMq to RabbitMq. We have around 12 endpoints and we are all in control of them. The endpoints are using sql persistence. Furthermore several endpoints are using timeouts.

We could not find a lot of documentation on when to use which migration strategy. The implementation of the Router.Migrator doesn’t seem to be appropriate for our situation because we are in control of all endpoints and our queues are not continuously filled with messages.

The best strategy seems to be to migrate all the endpoints at once to RabbitMq. When the (MsMq) queues are empty of course.

Is this assumption correct? Or are we forgetting other aspects?

NServiceBus 7.1.10
Persistence: SqlPersistence with dialect MsSqlServer
Transport: MsMq

#2

Hi @KoenMeijer

You can try to divide migration by independent processes/policies for example:

P1 -> start (user action/schedule action/…) -> handle -> … -> timeout -> … -> handle -> end
P2 -> start (user action/schedule action/…) -> handle -> … -> timeout -> … -> handle -> end
P3 -> start (user action/schedule action/…) -> handle -> … -> timeout -> … -> handle -> end
…

Migrate P1 first and check if everything works correctly. Then migrate P2 and check and so forth.
One step further is to divide process/policy by data (employees/customers/categories/…), migrate some part of them and check. Then migrate the next part and check.

The pros of this strategy is migration functionality one by one. The cons is that during migration you have to maintenance two systems (optionally with partition data) at the same time - MSMQ and RabbitMQ.

If you have “never ending” processes/policies with “never ending” timeouts you could try to use Migrating timeouts tool.

Hope this help.
Mike

#3

Koen:

Migrating them all at once when the queues are empty is fine as far as the transport is solely concerned.

The problem may be that you have been relying on distributed transactions (DTC) to maintain consistency between message completion, saga updates, and/or business data when using both MSMQ + SQL Server. Since RabbitMQ cannot participate in the transactions you will want to review your system to make sure you aren’t relying on DTC.

Either way, and especially if you are currently relying on DTC, you may want to look into our Outbox feature.

#4

Thanks for your message Mike. Our processes are “dividable” and are “ending” so the strategy is a definitely an option.

#5

Thanks Bob, we are relying on DTC now. The Outbox feature we can use for almost all our endpoints. With two endpoints we will get an issue because we can not change the structure of the database. So persistence (and adding the Outbox table) is not an option.
But, the endpoints will be obsolete at the end of the year.
We are considering to use these two endpoints without the Outbox feature, so with the inconsistency risk.
Or are there any other options?

#6

I think the only other option is to implement custom idempotent logic in all handlers hosted in these Endpoints.

Hope this help.
Mike

#7

@KoenMeijer the latest versions of SQL Persistence allow using TransactionScope for the outbox transaction.

This means that in these legacy endpoints you can configure the SQL Persistence to use some other database (the outbox database) and configure TrasactionScope outbox. Then in the handler you can open the connection to the business database and carry out the work in the same way as before.

The result is a distributed transaction spanning the outbox database nad a business database. One leg of that transaction contains the business changes and the other attempts to insert the outbox record, giving you the same level of guarantees as the current solution.

2 Likes
#8

That is great! Will work for us. Thanks!