Outbox with long running handler

Hi All
I was going through the official documentation of Outbox implementation, Below are the steps taken from the link: Outbox • NServiceBus • Particular Docs.
Here we are initiating the transaction at step no 3 & committing at step7. Consider a scenario where we my handler is a long running handler and takes several minutes to complete then does it means it would lock nServiceBus db during this time?

  1. Receive the incoming message from the queue.
  • Do not acknowledge receipt of the message yet, so that if processing fails, the message will be delivered again.
  1. Check outbox storage in the database to see if the incoming message has already been processed. This is called deduplication.
  • If the message has already been processed, skip to step 7.
  • If the message has not yet been processed, continue to step 3.
  1. Begin a transaction in the database.
  2. Execute the message handler for the incoming message
  • Any outgoing messages are not immediately sent.
  1. Store any outgoing messages in outbox storage in the database.
  2. Commit the transaction in the database.
  • This is the operation that ensures consistency between messaging and database operations.

Regards

Hi @brijmailid

Any series of steps that are enclosed within the scope of a transaction will be performed in its entirety and the DB that is selected will use its ‘Locking’ mechanism. This ensures that the transaction isolation and its durability is preserved. The outbox feature requires persistence in order to store the messages and enable deduplication.

Thanks
Jayanthi
Particular Software Support

Hi @Jayanthi_Sourirajan
Thanks for your reply but it is very generic around DB transaction & Locking. My question was very specific to the steps mentioned in the documentation. In my environment, I’m using SQL for SAGA data persistence & ASB for messaging

  1. Begin a transaction in the database.
    Trust this would be initiated in nServiceBus Db

  2. Execute the message handler for the incoming message

    • Any outgoing messages are not immediately sent.
      Are we referring to “Outgoing message” being generated during handler execution?
  3. Store any outgoing messages in outbox storage in the database.
    Store message generated within handler to outbox db

  4. Commit the transaction in the database.

Thanks

@brijmailid,

I recently wrote What's an Outbox and why do we need it? Hint: it's about data integrity
The article clarifies the outbox pattern and how it works in a messaging context.

Let me know if that helps or if any further clarification is needed.
.m

Thank you so much, I will have a look on the link you have provided. I understands the purpose of outbox but my question is more about the duration of lock that will be applied on nServiceBus DB.
Hoping to get some good insight around outbox :slight_smile:
Thanks

By default, the duration of the lock matches the default transaction duration.

Long-running handlers are problematic because the outbox transaction starts when the incoming message is received and stored in the outbox storage. The outbox transaction risks timing out before the handler completes its tasks. If that happens, the incoming message will be retried. It might result in an endless loop. It might be even worse if message processing has side effects that cannot be rolled back with the transaction.

A couple of articles on long-running handlers:

Thanks @mauroservienti, it clarified my doubt.

  1. Lock will be applied on outbox on row corresponding to the incoming message and it will remain in locked state during execution of handler.
    – Trust lock would be pessimistic & Transaction lock duration would be default.
    – I believe nServiceBus will not be able to bubble up exception to the executing handler in scenarios where it came across any issue while writing or committing transaction to outbox DB.
    – we have nServiceDB stored on Azure on SQL elastic pool & Application database is on On-Prim server. Is it a point of worry?

Thanks

Incorrect, only if pessimistic locking is explicitly enabled, default is optimistic concurrency:

Not sure what you are asking but the outbox persistence happens AFTER all handlers are completed. Recoverability will handle it which means the message will be retried or forwarded to the error queue.

No, SQL Azure is supported

The only thing you might want to consider is disabling outbox cleanup and have that done as a scheduled job: