Saga Azure Queue Storage gotcha?

1

We have a Saga batching stuff to send on completion, like this

image
image866×286 34.5 KB

This was running on Azure Service Bus (SendsAtomicWithReceive) with SQL Server, and everything worked fine. I then changed the transport to Azure Queue Storage (ReceiveOnly), and now we’re losing the last SomeCommand after the saga ends (the saga table is empty, but there’s no SomeCommand on the queue).

Is this expected? If so, why doesn’t it send the outgoing message and await the ACK before completing the saga?

(I will fix this by introducing Outbox)

2

You should complete saga with an explicit message and have a handler for that message that doesn’t do anything else.

The problematic scenario is when sagas are being completed together with sending/publishing outgoing messages. In case of failure after the saga is completed, the outgoing messages may not be dispatched. However, when the incoming message is retried the completed saga is not found, which results in outgoing messages being lost.

This issue can be avoided by:

  1. Enabling the Outbox feature, if supported by the chosen persistence.
  2. Ensure that no outgoing messages will be dispatched by completing the saga from a timeout or sending an explicit command to self.
  3. Replace saga completion with soft delete by setting a flag/timestamp and use some native mechanism of the selected storage to cleanup old saga instances.

Source: doco

ASB transport is prone to this problem as well, SendsAtomicWithReceive transaction mode has probably made it less obvious.

3

Thanks, makes sense. So you delete the saga state as the first step.

4

@SeanFeldman could you also comment on this solution?

image
image740×384 41.8 KB

5

No. You issue a command to complete saga as the last step. And not as an immediate dispatch.