How to add message handler results to central list?


I’m wanting to use NServiceBus’s concurrency to plow through a repetitive action and append the result to an list in memory. I have a list of ID’s and I want to perform an operation on each of the IDs, and make a list of the results to insert into a database. I was thinking I would have a secondary endpoint with some concurrency > 1 that would handle a message with one of the IDs, and an object would be sent back in the reply, and that object would be added to a list. When the list is complete, I would write the list to the database in a single connection.

I was hoping I could use this approach instead of writing each result to the DB as it is calculated so I wouldn’t waste all the extra time opening and closing the connection for each row, and also to avoid database deadlocks.

Is this possible to implement in NServiceBus? It seems it would be straightforward with this kind of tool. The part I’m getting hung up on is once the calculation is performed for the ID, how do I reply in a way that in my handler that sent off the message with the ID can add the result to the list for each calculation? Thanks.

(C#, .NET Standard 2.0)

Hi Steve

To me that sounds like a use case for a saga. With sagas you can build a stateful entity that tracks the progress of the list and when it is done sends a commend to same endpoint (SendLocal) to write the result into the database as a single transaction.

Have a look at our saga getting started guide

There is one caveat though. The saga itself if the state needs to be persistent over restarts requires a persistence that will also manage database connections. But at least the connection doesn’t have to be opened for the long time of the work. But every time the saga “ticks” of an id from the list the saga state will be written back to the persistence and therefore IO occurs. If you want to avoid this and can afford to lose the work in progress state you could plug in the InMemory saga storage for that endpoint with the drawback of losing all the saga state when the endpoint is restarted.