We are using CosmosDB persistence and therefore have to configure how the partition key should be determined for incoming messages. Sometimes we want the same event to be handled by two different handlers writing data to different partitions. What would be the best way to handle the same message in the context of 2 different partition keys?
Our first idea is to add a ‘EventDuplicator’ handler to our endpoint. For example for event Abc the partition key is determined based on Property1 and is handled by Handler1. The MessageDuplicator also handles event Abc, copies the data to event AbcForPartition2 and sends locally. For event AbcForPartition2 the partition key is determined using Property2 and it is handled by Handler2.
A specific example is a Saga. Its data is automatically stored in the partition determined by the incoming message, which means all events intended for a Saga always need to have the same partition key configuration. This is sometimes difficult to achieve and the requirement is not very obvious.
Are there any better/simpler ways to handle this? Is there perhaps a way to use an NServiceBus behavior we could use to achieve this?
Can you explain that further? The data is not necessarily “automatically” stored by the incoming message. By default, the saga persister falls back to using the saga ID as a partition key which is a deterministic ID derived from the saga entity full name, the correlation property name and the correlation property value. It is possible to override this though to align those two cases if needed.
So you would like to have two handlers in the same endpoint handling the same message but using different partition keys? Is my understanding correct?
Currently there is a one to one relationship between a message and a partition key. What you seem to need is a way to have multiple partition keys for the same message and that is not supported. So indeed you would have to treat this case as “local transactions / commands”. I would see it as the following: When the event Abc comes in that event is not really the carrier of the partition key and for internal purposes you need to split that up into two local commands that then triggers the logic that is associated with a partition key. The benefit of this approach is that one of the commands can be directly handled and associated with the saga instance.
You could try to avoid one of the local commands by doing the cosmos writes directly in the event handler for Abc but to me it feels cleaner to seperate both concerns into dedicated handlers but it is hard to tell by these very generic examples you have given. An additional benefit of having two local commands is that those individual transactions can be named after their business intent and will be visible in the message conversation and can be retried independently if needed.
Without knowing more about the actual use case I doubt there is another way because of the association with message => partition key and the partition key extraction mechanism has no way of differentiating those two cases.
About saga storage: we have not seen the fallback you described. Maybe because we also have Outbox enabled and the partition key value is already determined in a different way before the Saga can derive a partition key?
Your understanding is correct. I like your idea to split the event and give the resulting messages a business meaning instead of just duplicating the event for technical reasons. Thanks for your suggestion.
Yes, with the outbox the behavior is different but you can still control the partition key with the transaction APIs based on the headers (physical stage) or information from the incoming message (logical stage). It is just that by default we set the Partition Key to the message id of the incoming message for convenience reasons so that things work “out of the box”.
