Leveraging Outbox consistency mechanisms outside of the context of a message handler


(Edward Dewe) #1

First some background:

We are in the process of removing a home grown web api/messaging ‘synchronisation’ mechanism from our NSB based platform. This used custom message headers, a process id and a DB polling mechanism to allow us to support 200 style ‘synchronous’ web api calls over the asynchronous message based back end.

Before I go any further I will make it clear that we have long had this flagged as an anti pattern - it has just been so deeply embedded in our older API that it has been a challenge to factor it out till now.

To replace this we have implemented a command ‘executor’ pattern in our legacy web api to allow us to ‘handle’ commands synchronously in the controller code where absolutely necessary. Where this results in events being fired we are leaning on a shared subscription store so that the web api can publish events on to the appropriate consumers.

This is all going swimmingly - performance of the SPA apps that consume this legacy API has improved dramatically - the code is a huge step closer to a vanilla NSB implementation etc etc…

But there is one downside - we loose the transaction scope/unit work around these commands we handle directly within the web api. Events can go out ahead of business data fully committing to the DB, or worse could go out when the associated business data has failed to commit. At present we have had to accept this as the cost for the benefits we gain in eliminating the complex synchronisation mechanism, and it is a trade off we are quite prepared to make.

However, I want to know if the Particular team have made any progress towards supporting the outbox shared session scope outside the context of NSB message handlers - i.e. make it possible for these kinds of web api hosted business operations to be supported?

I found this page from awhile back with some suggestion of a solution - but the thread is cold: https://groups.google.com/forum/#!topic/particularsoftware/rF11h7ebHF4

Jeremy Miller has tackled this with his implementation of outbox for his Jasper framework - he’s clearly been informed by NSBs outbox work but he has made the mental leap which decouples the outbox mechanism from a hard dependency on the handler running in the context of a back end worker: https://jeremydmiller.com/2018/04/16/jaspers-outbox-pattern-support/

Any thoughts?


(Ramon Smits) #2

Our outbox still has the same restrictions. We do have implementation ideas but currently no working solutions.

Some alternatives:

  • When using MSMQ, use distributed transactions but this does take a dependency on MSDTC.

  • Put your DB I/O and message sending in a handler, and invoke this handler from your current code.

  • In some scenario just setting a message ID to a deterministic value removes the need to use an outbox as long as your processing is idempotent.

  • Instead of invoking Send or Publish immediately, put these in a collection first with a deterministic ID, then after the DB operation committed send this collection of messages.

If these options are not possible then I’m afraid you currently have to enroll your own outbox pattern like store.

– Ramon


(Edward Dewe) #3

Hi Ramon, Thanks for your response.

We are on AWS SQS (competing consumers) + Raven. DTC would be a backward step for us.

I’m not sure I understand your second comment - you mean execute them in a handler in the context of the web api or push them back on to the bus (where they used to be). If it is the later that is certainly our fall back on a case by case basis where the need for consistency outweighs the synchronisation issues.

In that case we would use specific sagas (as opposed to the blanket custom mechanism we had) to tie together the overall process status and the API or client would revert to a polling strategy, or hook into client side notifications (we are starting to use PubNub for this).

This is ultimately the route we will go down where the specific circumstances warrant the coordination - it will be a bit of a journey though as we have 100s of command handlers triggered via the API.

I do think you guys should consider promoting the outbox ‘session’ so it can be used in either a bus or an api context (or perhaps support in ‘in process’ flavour of send local to handle initial commands synchronously. At the end of the day they are both just different flavours of application hosting processes - supporting different ‘inbound’ communication protocols.

Cheers,

Ed


(Karl) #4

Hi,

We have exactly the same request, it would be very benificial to be able to use the outbox mechanism outside of a handler context. When you want the UI to be consistent as soon as possible, you need to work synchronously. Having an in-process SendLocal and handler that leverages the Outbox functionality would solve this. It’s not about the deduplication of messages, but for the transactionality it provides : saving business data and sending/publishing messages in one transaction.

Thanks
Karl


(Ramon Smits) #5

Could you please share which persistence and transport you are using?

Would it be an option to just do a SendLocal, where the handler does both the manipulation of the business data and a Publish and a Reply?

Imagine the database is slow, returns a deadlock exception, results in a timeout, or is even unavailable. How would you deal with this in this blocking context?


(Karl) #6

We are using the Azure Service Bus transport and SQLPersistence.

Using a SendLocal will go back to the bus, and means working asynchronously. Which delays the rendering of the UI. I agree that working asynchronously has a lot of benefits, such as the retry-mechanisms. But sometimes rendering the UI synchronously is preferred over the benefits of working asynchronously.


(Ramon Smits) #7

So how about a slow or unresponsive database?

I think that is you really want to have your process synchronous that that does not mean you cannot use asynchronous communication.

If you would use a handler that does both a publish + reply and combine that with callbacks that you have the best of both worlds:

Imagine that you would have an outbox that does supports it could still occur that the database is slow or unresponsive.

In the case with callbacks you can at least have it have a best effort including retry logic and even if the client disconnects or does not retry that the task will eventually be done.


(Karl) #8

So, if I understand correctly:

  • first do a SendLocal of a command from/to the web api endpoint
  • the commandhandler saves the business data to the database, publishes an integration event and replies to the sender (the web api endpoint itself). If we enable the outbox in the web api endpoint, these three actions will be done transactionally and will leverage retries.
  • the web api receives the reply either in a callback or in a handler and sends the HTTP response to the caller of the API

Is this correct? Or am I still missing something? :slight_smile: