Efficiently Ignoring Messages

1

Scenario: A subscriber subscribes to an event type, but in practice actually ignores all but 5% of the messages.

This seems at the very least wasteful, but at the worst a problem that could cause MSMQ quotas to be exceeded, affecting other services using MSMQ.

What are suitable/NServiceBus-idiomatic ways to resolve this?

Ideas include:

  • Don’t worry about it, as long as you’re always careful to ignore messages for the simplest reasons first.
  • Filtering at the publisher on more than just event type. I don’t like this as it couples publisher and subscriber and effectively makes it a broker system where the thing in the middle (publisher) knows too much.
  • Create more specific message types somehow. Again feels like coupling as you’re changing the publisher to suit a specific subscriber.
  • Topic-based subscriptions (which would have to be hand-rolled as NSB doesn’t support them as such, though I’m sure hooking into the subscription control message sending/handling and doing something in the headers could be made to work).
2

Hi Neil,

I would not worry about it too much. Go for the simple approach which is ignoring messages with simple reasons as a first operation and then they are consumed. Storage is cheap. If you can solve a problem with provisioning more storage if needed instead of going for a more complex filtering solution I would always go for the simple one that can be mitigated with hardware. My 2 cents

Regards
Daniel

3

I think this can lead to more explicit (and better) message contracts if the filtering is legit from a business perspective. Can you elaborate on the business requirement that you have in this specific case?

4

Imagine a scenario where a publisher publishes “PackageDelivered” events. One of the subscribers needs to update SalesOrders when all their packages are delivered, but not all Packages were even picked from the warehouse (some came from 3rd party warehouses, for example).

So the only event to subscribe to is PackageDelivered, but perhaps only 50% of those are relevant to the subscriber in question. The rest are ignored because there is no SalesOrder in that system relating to them.

There’s not really any reason other than sheer performance/avoiding waste for the publisher to know this before publishing. Though it could be achieved, it’s pretty nasty coupling.

5

One of the subscribers needs to update SalesOrders when all their packages are delivered, but not all Packages were even picked from the warehouse (some came from 3rd party warehouses, for example).

When you say “their” packages, does it mean that there are other service that also ship things or are “their” packages special in any way?

Perhaps that could be modeled with the message contracts?

class XyzPackageDelivered : PackageDelivered

or could it be that there is some piece missing that would be aware of the entire shipping process that can figure out if the entire order is shipped?

class ShipmentComplete
{
    public string OrderId;
}

In general this feels a little like content based routing, Udi has a good write up on that topic that might give some ideas as well:

http://udidahan.com/2011/03/20/careful-with-content-based-routing/

6

Wow I had no idea I’d actually had replies to this - so sorry I didn’t come back to you.

I think perhaps what I would end up doing in the case I described is create the following:

class PackageDelivered {}
class WarehousePackageDelivered : PackageDelivered {}

And so the service that only cares about items picked from the warehouse would subscribe to the more specific event.

However, if I had a service that was customer-specific, I’m back to content-based-routing. Perhaps a subscribed that is bespoke in some way, sends an email for PackageDelivered events, but only packages delivered to a specific set of postcodes. You can’t do that purely with the type of event.

Again I think it comes down to - hope it’s not a problem and/or throw hardware at it (in the case of MSMQ, put the subscriber on a VM on it’s own so it has all the MSMQ quota it wants without affecting anything else.