Efficiently Ignoring Messages


(Neil Barnwell) #1

Scenario: A subscriber subscribes to an event type, but in practice actually ignores all but 5% of the messages.

This seems at the very least wasteful, but at the worst a problem that could cause MSMQ quotas to be exceeded, affecting other services using MSMQ.

What are suitable/NServiceBus-idiomatic ways to resolve this?

Ideas include:

  • Don’t worry about it, as long as you’re always careful to ignore messages for the simplest reasons first.
  • Filtering at the publisher on more than just event type. I don’t like this as it couples publisher and subscriber and effectively makes it a broker system where the thing in the middle (publisher) knows too much.
  • Create more specific message types somehow. Again feels like coupling as you’re changing the publisher to suit a specific subscriber.
  • Topic-based subscriptions (which would have to be hand-rolled as NSB doesn’t support them as such, though I’m sure hooking into the subscription control message sending/handling and doing something in the headers could be made to work).

(Daniel Marbach) #2

Hi Neil,

I would not worry about it too much. Go for the simple approach which is ignoring messages with simple reasons as a first operation and then they are consumed. Storage is cheap. If you can solve a problem with provisioning more storage if needed instead of going for a more complex filtering solution I would always go for the simple one that can be mitigated with hardware. My 2 cents

Regards
Daniel


(Andreas Öhlund) #3

I think this can lead to more explicit (and better) message contracts if the filtering is legit from a business perspective. Can you elaborate on the business requirement that you have in this specific case?


(Neil Barnwell) #4

Imagine a scenario where a publisher publishes “PackageDelivered” events. One of the subscribers needs to update SalesOrders when all their packages are delivered, but not all Packages were even picked from the warehouse (some came from 3rd party warehouses, for example).

So the only event to subscribe to is PackageDelivered, but perhaps only 50% of those are relevant to the subscriber in question. The rest are ignored because there is no SalesOrder in that system relating to them.

There’s not really any reason other than sheer performance/avoiding waste for the publisher to know this before publishing. Though it could be achieved, it’s pretty nasty coupling.


(Andreas Öhlund) #5

One of the subscribers needs to update SalesOrders when all their packages are delivered, but not all Packages were even picked from the warehouse (some came from 3rd party warehouses, for example).

When you say “their” packages, does it mean that there are other service that also ship things or are “their” packages special in any way?

Perhaps that could be modeled with the message contracts?

class XyzPackageDelivered : PackageDelivered

or could it be that there is some piece missing that would be aware of the entire shipping process that can figure out if the entire order is shipped?

class ShipmentComplete
{
    public string OrderId;
}

In general this feels a little like content based routing, Udi has a good write up on that topic that might give some ideas as well:

http://udidahan.com/2011/03/20/careful-with-content-based-routing/