It is slightly confusing to them as it mixes Inbox and Outbox concepts.
It assumes subscribers have access to the publisher’s outbox store. That is almost always blocked to possible avoid database integration.
The publisher and subscriber could be on entirely different database technologies.
It removes the message from as soon as it gets dispatched. This behavior is not configurable. We want to keep the messages for days and clean them out or move them to archives. This may sound like a precursor to event sourcing.
It is slightly confusing to them as it mixes Inbox and Outbox concepts.
You are right, technically it is both Outbox and Inbox. While the name is not technically 100% accurate, we believed that is OK to call it that way because the focus is more on the Outbox side. The decision was
It assumes subscribers have access to the publisher’s outbox store
No, it does not. Sender and receiver (or Publisher and Subscriber) can (and should) use different storages for the Outbox. We made a mistake in NHibernate persistence to default to a single Outbox table for the whole system and we can’t easy change that without breaking compatibility but the newer SQL persistence has a better default where the outbox table is prefixed with the name of the endpoint. These tables can be hosted in different DBs as long as the same DB is used for the Outbox and for the business data
The publisher and subscriber could be on entirely different database technologies
Yes, that’s possible
It removes the message from as soon as it gets dispatched
That’s true. The current implementations optimize for storage space and remove the messages as soon as they are dispatched. This is because the Outbox needs to be stored in the same DB as business data and we believe that this is not the best place to store message archives as it impacts the performance of business data access. A better solution is to use the Auditing feature that sends a copy of each processed message to the audit queue. This queue is processed by ServiceControl and the messages are stored in ServiceControl internal database for a configured period of time (by default one month). If you wish to store the data in a different format you can build your own audit processing endpoint and either attach it to the audit queue directly or use ServiceControl audit forwarding and attach it behind service control (to the audit.log queue)
Can you give an example of how the different database technologies work with NSB Outbox?
I get the audit queue usage. Please correct me if I am wrong but you get the audit copy when the message was is processed by the receiver. So, it is useful on the receiver side. I don’t think you get the same copy on the sending side, do you? The outbox table is the best place I can think of for the sending side because the intent of sending the message and its body are captured even if the transport is not available. I agree that business data and archivable/journal data shouldn’t be together. I have other processes to take care of that. I also may not be able to use or want transactions with that system. It is the most trustworthy place because that data was saved in the one business transaction in the same database. You can’t do this well if the transport is in RabbitMQ and data is in SQL. Are there any other technical difficulties in making it configurable?
Can you give an example of how the different database technologies work with NSB Outbox?
You can have a sender that uses NHibernate persistence and stores its outbox data in a table in a SQL Server and a receiver which uses MongoDB persietence and stores its outbox in the MongoDB instance. They can communicate without any problems.
I don’t think you get the same copy on the sending side, do you?
No, the audit works on the receiver side.
Can you raise an issue in the SQL Persistence repository and describe your case? I’ll make sure it is triaged but I can’t promise when we get to implementing this. In the meantime, I think you can use instead of triggers in SQL Server to prevent removal of messages when the dispatch happens. There is only one type of update issued against the Outbox table and this is the one that removes the information about the message. You can replace that update by adding an instead of trigger that just sets the dispatched flag and dispatch timestamp. Alternatively you can have an after update trigger that inserts the message body into a separate audit table.
How does this work in practice across service boundaries then? If the inbox and outbox are shared, is the consumer’s inbox only applicable within a service boundary? I didn’t think consumed messages were inserted into the outbox for idempotent receivers - that it was only marked as dispatched. It’s not clear from the docs how a cross-persistence scenario would play out, I don’t see anything that records messages once they’re processed for future de-dupe checks.
I don’t see anything that records messages once they’re processed for future de-dupe checks
The outbox ID is the id of the processed message and that is kept for de-duplication. The outgoing messages column is cleared to conserve space in the DB.
The outbox requires an incoming message context. Use the IMessageHandlerContext instance to dispatch messages handled by the outbox.
What about the use-cases where we generate messages from the legacy batch processing logic? We would like to also make sure that the messages are sent in a transactional manner, regardles of the origin of the message.
We will switch from the batch processing to the event based eventually, but this limitation makes it harder to migrate to NServiceBus and maintain the transactional security of the messages. This forces us to build our own outbox(es) to enter the NServiceBus game.
Can we use the outbox feature without the incoming message handler?
@bosko.stupar, it’ll be available. We’re currently working on the NServiceBus.TransactionalSession package that will enable outbox-like behavior with all the supported persisters outside the context of an incoming message.
I don’t have details about its work-in-progress status at the moment.