NServiceBus in a single-process application?

1

Hi,

First off, I am quite new to asynchronous messaging, and to NServiceBus in particular. That said, I am investigating viability of a technical architecture for a client.

The application in question is a single-user, background service that is installed on a single appliance in an industrial environment (or on a client computer for testing, demo’s) running on Windows 10. It accepts sensory value samples on a tcp port, and sends a processed message of those samples on to a display or client application.

Inside the application are 5 ‘building blocks’ each with their own responsibility to handle the incoming values. The plan is to implement it as a single process (Windows service) to keep installation and troubleshooting straightforward.

Included is a diagram of the data flow and building blocks. Samplebox and Display/Meltcontrol are external appliances/applications. The application in question is inside the fat rectangle.

image
image945×493 16.5 KB

I have implemented a PoC as a Generic Host that spins up five BackgroundServices, one for each building block, and am trying to use NServiceBus to handle intra-process communication asynchrously. Since the application already requires SQL Server, I am reusing SQL as Transport. This eliminates dependency on- and installation of yet another layer such as RabbitMQ.

So far that seems to work, but I can’t solve following issues:

  • Prevent data loss. Pick up where we left of in case the process or one of its blocks dies. Synchronous communication looses the sample value in that case.
  • Scalability. Launch more than one instance of the block with the highest workload. Implement the competing-consumer pattern.

I think this is not a typical use case for NServiceBus, or at least I cannot find all the info I need on SO or in the docs. So I would like to ask for some help:

  • In several articles/posts I read that restarting BackgroundServices that rely on EndpointConfigurations causes issues because of the way endpoints are tied to the DI container. That would defeat the purpose of preventing data loss.
    Is there a better way?

  • My idea to scale out was to spin up several instances of the BackgroundService with the highest workload, and have it start the endpoint, but I saw in my PoC and read here that only the first hosted service is registered when calling AddHostedService multiple times for the same class.
    Is there a better way to implement the competing-consumer pattern in this architecture?

  • Your opinion on the proposed architecture? It seems to me that running each ‘block’ in its own process (Windows service) and implementing a solid Setup.exe to get them installed, would be less complex and less time-consuming than getting this to work?

  • Pricing. Since every customer would run its own instance of this application, I guess the Ultimate license is our only option?

Thx a bunch in advance.

2

In my experience trying to cram multiple processes inside a process boundary is always more trouble than it’s worth. Processes come with their own system-level tools for knowing if they die and dealing with it. Pack them together in one physical process and suddenly that’s your problem.

It can be done, but it’s difficult. You have to be very specific about assembly scanning. If you have to “restart” an endpoint, you can’t reuse the EndpointConfiguration (the DI issues you mentioned) but you can stop an endpoint, recreate a new EndpointConfiguration with the same settings, and then start that.

I’m curious what issues you’re having with data loss. You’re using queues. (Well, database tables abused (by us, and we’re very good at it) as queues.) Anything in a queue is safe. Anything that fails will be retried.

Scaling out within the same process (or even on the same server) is pointless. What you’re looking for is LimitMessageProcessingConcurrencyTo to scale up on the same hardware. An additional process on the same host is no better than doubling the concurrency limit, just more complex. Scaling out is for additional machines.

Absolutely agree that separate services with an installer is better in almost all cases.

Pricing: If you’re developing a reusable system that you install at multiple client sites, then you’re acting as an ISV and you’re correct that you’re talking about the Ultimate tier. If this is bespoke software for one client and you’re acting in a consultant capacity then it’s not necessarily Ultimate. I’d encourage you to talk to our licensing people. Click the Contact Us link or open the live chat on this page.

3

Thx, @DavidBoike, for the reply.

The issue I have with preventing data loss pertains to the fact that I can’t simply restart a BackgroundService (or rather, the EndpointConfig in it). I understand that “Anything that fails will be retried”, but only if the failing process ánd the endpoint it gets data from, can be restarted.

Thanks for the tip about setting the concurrency limit. That will definitely make things easier.
(And yes, I meant scale up obviously. My mistake.)

Still heavily considering separate Windows services.

I am only writing this PoC for a manufacturer of appliances. If it gets accepted, every appliance will hold a copy of the app including NServiceBus. So definitely Ultimate Tier.
Licensing is up to the manufacturer; I only need to inform them what the cost would be.