I’m trying to decide whether NServiceBus is a good fit for a current problem that I’ve encountered. In essence, my problem is about testing a solution, rather than how to implement one.
I’d like to decouple an existing algorithm into an asynchronous, message-based backend. The reason is that the algorithm, as it’s currently defined, queries a lot of online data in a doubly-nested loop. The current implementation is, however, a blocking procedure that runs for about ten minutes before it returns. It’s a data processing algorithm, rather than a business logic implementation.
I have a fairly good idea how I could implement the algorithm with NServiceBus’ Saga feature. An initial message could trigger a workflow that produces a few hundred new messages, which then produces a dozen more messages per message, and so on. I know that this sounds like something that grows polynomially, but the algorithm has limits on both levels of nesting as well as the number of messages. In terms of message count, this is really an O(1) algorithm, although, and herein lies the problem, ‘1’ is rather large: On the order of a few thousand messages.
So, to be honest, I’m not entirely convinced that NServiceBus is the best-suited platform for this kind of problem, or if I should rather look towards some kind of map/reduce system. It’s just that, while this algorithm seems a bit too ‘busy’ (in terms of message count) for a typical NServiceBus scenario, on the other hand, it strikes me as ‘too small’ for a map/reduce infrastructure. But I could be wrong. That’s why I write here.
In any case, my main concern is how to test a hypothetical NServiceBus-based implementation. I already have working tests of the algorithm in question. Since I use a Fake object instead of out-of-process queries, and the tests run with smaller data sets, I can run through a few hundred tests in a couple of seconds.
As a proof of concept, I reimplemented the algorithm with Reactive Extensions, and kept the externally visible method as a Facade. This works just fine, since I just let all the messaging run in memory.
Can I do something similar with NServiceBus? I’d like to keep my existing tests unaware of implementation details, and I would, in this scenario, consider NServiceBus an implementation detail.
It seems to me that what I’d need is in-memory transport and persistence. It looks as though perhaps the non-durable persistence might fit my requirements, but I can’t find a similar non-durable transport.
I’ve been looking through the example code documented in Saga scenario testing, but these tests seem too low-level for my requirements. As far as I understand, I need to ‘manually’ trigger message handling using HandleQueuedMessage, and since I may have thousands of messages, that doesn’t look like a robust way to modify existing tests.
I’m also aware of the forum thread End-to-end / Integration testing, but it doesn’t seem to include any content relevant for the scenario that I’ve outlined here.