Hi all,
Microsoft Azure recently made some changes to their Service Bus “Standard Namespace” settings that resulted in our namespaces all being throttled heavily. Fun part is that they did this in the afternoon on Friday before Easter! After several days of trying to get in touch with them, they finally got someone to change our production namespace back to the previous settings while we migrate to premium. The process looks pretty simple except for moving messages from the old namespace in to the new namespace which is step 5 in the document below:
I have two questions right now, I guess.
Has anyone here had any good or bad experience with this process?
For moving the messages, I was going to write something that did it directly against the azure queues/topics, but wanted to check to see if there was a better (NSB) way to do it?
The quota based throttling was introduced a while ago. There was a bit and the credits were not informed correctly. The issue was fixed and a heavy load users on standard tier started feeling the impact (over 1000 credits/sec).
I haven’t done this in a while but you shouldn’t be moving any messages found into the entities unless those were received while the migration was ongoing. For those, it would be receiving, cloning, and sending. Since there’s no spanning transaction, I’d ensure you don’t receive and complete but peek lock, to dead letter if anything goes wrong.
All for fixing bugs, just not on Fridays and/or holidays. We definitely should have been on premium so that’s on us.
Good call on the dead lettering. I am having issues pulling back scheduled messages unless I do just a peek but then I have no way to resolve them because there is no lock/whatever it wants.
The following works for scheduled and regular but leaves them all in the queues which will be annoying if I have to rerun something. Still looking but if you have any thoughts, I’d appreciate it. I am probably going to do this just for scheduled messages and do the peek/lock/deadletter for normal messages.
var premiumSender = premiumMessageClient.CreateSender(queueName);
var originalReceiver = originalMessageClient.CreateReceiver(queueName);
var receivedMessage = await originalReceiver.PeekMessageAsync();
while (receivedMessage != null)
{
var newMessage = new ServiceBusMessage(receivedMessage);
await premiumSender.SendMessageAsync(newMessage);
receivedMessage = await originalReceiver.PeekMessageAsync();
}
Will try more in the morning with fresh eyes.
Thanks,
Nate
If the messages are already scheduled, those should be migrated along with the namespace. Unless you want to do it manually. The only way to “complete” those would be too cancel scheduled messages by their sequence number. To get the sequence number, you’ll have to pick the messages first.
Well that would make things easier! I guess I should actually run a migration and see. I was just trying to cover my bases before running one. This works but doesn’t do the deadletter thing.
var premiumSender = premiumMessageClient.CreateSender(queueName);
var originalReceiver = originalMessageClient.CreateReceiver(queueName);
var originalSender = originalMessageClient.CreateSender(queueName);
var receivedMessage = await originalReceiver.PeekMessageAsync();
while (receivedMessage != null)
{
var newMessage = new ServiceBusMessage(receivedMessage);
await premiumSender.SendMessageAsync(newMessage);
if ( receivedMessage.State == ServiceBusMessageState.Scheduled)
await originalSender.CancelScheduledMessageAsync(receivedMessage.SequenceNumber);
receivedMessage = await originalReceiver.PeekMessageAsync();
}
After several days of trying to get in touch with them, they finally got someone to change our production namespace back to the previous settings while we migrate to premium. The process looks pretty simple except for moving messages from the old namespace in to the new namespace which is step 5 in the document below: