Sending big batches of messages

We’re using the Azure ServiceBus transport, but I think the underlying topic could apply to all transports.

When there’s certain events happening in the system, we sometimes need to send huge batches of messages, in the range of thousands or tens of thousands. Think notifying all users in a group with a push notification, for example.

The starting point of this is often a single action/transaction (adding something applicable to all users) that we then need to fan out into individual, potentially personalized, messages sending an email, push notification, whatever “fan-out” method is needed.

We currently use a bit of a magical DataBus property (add unlimited ids) and keep batching this up into small chunks to stay under the Azure Service Bus limits of 100 messages, until it finally is a single message and we can process it with business logic.

Though this works great, is actually super fast it feels like a hack for something that should have a better process (or, if not, some helpers built-in that assist with this fan-out principle over all transports in a reliable manner, for example not running into limits of 100 messages).

Any thoughts or suggestions you might have would be more than welcome.

Hi JP,

This is a good question. Off top of my head, I can say that this problem is exacerbated by the underlying transport. If you look at each transport, each has its caveats. Batch size included. The 100 messages limit is there because you’re sending messages atomically. Remove the atomic guarantee (or the transactional guarantee that spans the incoming and the outgoing messages) and you’ll be able to send way more messages.

NServiceBus does provide a way to dispatch way more messages when using the outbox feature. That’s because all the outgoing messages would be dispatched using immediate dispatch and not batched. This would mean running with a persistence that supports the feature. Right now it’s SQL persistence, Mongo DB persistence and Cosmos DB persistence. If you’re on Azure Storage persistence, you might see some changes in the future or migrate to Cosmos DB persistence.

Hope that helps.

Specifically for ASB, the number of messages that can be sent in a single batch is also limited by the total batch size. But the size is not the primary limiting factor. It’s the ability of the broker to guarantee the atomicity of the operation if something goes wrong. If you want to stay with higher transport transactional guarantee and have a large batch size, you can ask the broker team if they are going to increase that limit.

Hi Sean,

Great, thanks for that. Just to clarify, does using the Outbox and the atomic sending combined mean we’d be able to send say 50,000 messages in one go, totally bypassing the 100-limit and blazing far past it? We use the SQL Saga Persistence already, so adding Outbox would be fine.

I think expanding the batch size from 100 on ASB to essentially limitless seems like a big architectural ask for the broker team, but I gave it a shot anyway: Increase batch size · Issue #396 · Azure/azure-service-bus · GitHub

does using the Outbox and the atomic sending combined mean we’d be able to send say 50,000 messages in one go, totally bypassing the 100-limit and blazing far past it?

No. The 100 limit is enforced by the ASB broker when using the highest transport transactional guarantee. When using the outbox feature, the highest transport transactional guarantee is not needed. The only caveat is that outbox can generate duplicates and if the destination is not de-duplicating (using the outbox as well), idempotency needs to be implemented manually.

I think expanding the batch size from 100 on ASB to essentially limitless seems like a big architectural ask for the broker team, but I gave it a shot anyway.

It’s a big change indeed. I cannot speak on behalf of that team :slightly_smiling_face:
Since you’re already using SQL persistence, you might be closer to the solution than you think.