We are currently having an issue where events never gets sent out from the outbox. They are left with Dispatch set to 0. We suspect this is because of our recoverability custom policy. According to my understanding, if Phase 2 of the outbox flow fails, the message has to be processed (as deduplicated) again for the event to have a chance to be sent out. Also, as far as I understand, if Phase 2 fails, we will trigger our custom policy. Our custom policy discards the message in case an error occurred.
In our handler, discarding a message is OK if anything goes wrong. However, what we did not realize from this custom policy was that we also seem to discard messages that successfully change the state of our system, without allowing events to be sent out.
First, do you agree that this would be the case of our custom policy? Second, is this expected behaviour? Third, could we somehow check if the Phase 1 transaction succeeded in the custom policy, and only in that case go for a retry?
Let me verify if I understand correctly. Your requirement is to have a message handler that processes incoming messages and publishes event. In case the handler fails (for any reason), the incoming message should be dropped and the handler should not be executed again. In case it succeeds, you want to ensure the published events reach their destinations.
If that’s the case then you are right, a custom policy that drops the messages in case of failure would prevent the proper dispatch if a failure happens during the dispatch phase.
One way of solving it would be wrapping the handler execution in a try-catch block and re-throwing a specific exception e.g. DoNotRetryException. Then, in the custom policy, you should be able check if the exception is of that well-known type and drop the message only in that case.
If you happen to have multiple handlers with similar requirements, you might consider adding the try-catch block in a pipeline behavior so that it is reusable.