Need to pause/delay message processing


(Alexander) #1

We’ve got a new requirement for an endpoint currently processing messages in a straight forward manner. The requirement is to delay processing all messages while batch cycle runs. After the batch cycle ends, the endpoint is to process the accumulated messages and resume normal straight forward message handling. The endpoint can subscribe for the batch cycle Start and End events.
What are good ways to do that?
Thanks


(Andreas Öhlund) #2

One option is to create a “controller” endpoint that subscribes to the start/stop batch events and starts and stops your processing endpoint accordingly.

Simple but it works :slight_smile:


(Alexander) #3

It might work. However, then the stopped endpoint will loose it’s heartbeat and this will cause panic in our support. Is there a way to handle heartbeat?
Are there other ways to pause?


(Andreas Öhlund) #4

You should be able to start the endpoint as SendOnly during the batch runs, then it will emit heartbeats while still not processing messages

https://docs.particular.net/nservicebus/hosting/#self-hosting-send-only-hosting

Hope this helps!


(Daniel Marbach) #5

Hi Alexander

Can you elaborate a bit more on that requirement? For me that sounds like a classical workaround that has been given to you by the business because the think about the solution upfront instead of given you the problem and let you guys figure out how to solve it.

Regards
Daniel


(Alexander) #6

Nice to see all the familiar faces - both Daniel and Andreas

I’m not sure what other details you need, Daniel.


(Alexander) #7

How do you suggest the service to know if it has to start as SendOnly mode? Would controller maintain this state and the service would reach out to it on its start?


(Andreas Öhlund) #8

How are you hosting the processing endpoint? Windows service?

A command line arg might be the easiest way to achieve this?


(Alexander) #9

Yes, I’m hosting it as windows service. Have not tried starting the service from another service, let alone pass dynamic command line arguments while starting. However, looks like it’s possible: https://stackoverflow.com/questions/18591048/how-to-pass-some-parameter-to-windows-service-when-run-programmatically

Can the endpoint have two queues - one for regular messages, and another (high priority) for controlling it?

Are there any other alternatives?


(Andreas Öhlund) #10

Just start and stop the processing service based on the “batch state changes”?

if(BATCH_START)
   net stop processor
   net start processor --send-only //or however you can pass cmd line args


if(BATCH_STOPPED)
   net stop processor
   net start processor

(Daniel Marbach) #11

I was wondering where this requirement came from and why? Sometimes understanding that might allow us to give you another solution that fixes the cause why the business formulated this workaround as a requirement. Or maybe I misinterpreted the word requirement.

Daniel


(Alexander) #12

I hear what you’re saying, Daniel. I don’t want to go into details but we’re researching other options as well and want to have them all on the table to make the final decision on the approach we will take. In this thread, I want to find possible technical solutions to the problem stated.


(Alexander) #13

Can the endpoint have two queues - one for regular messages, and another (high priority) for controlling it?


(Alexander) #14

still need an answer to this:
Can the endpoint have two queues - one for regular messages, and another (high priority) for controlling it?
Thanks


(Andreas Öhlund) #15

You could use a “Satellite” to acheive this

https://docs.particular.net/nservicebus/satellites

Although I would still prefer to use a separate endpoint to control this like suggested above


(Alexander) #16

After some messing around with other ways, it looks like I’m going to try to implement your suggestion - have a controller stop the service.
Is there a good reason for not having an NSB functionality similar to this - ability to stop reading the queue without shutting down the endpoint?


(Andreas Öhlund) #17

It has come up now and then but so far it seems that stopping and starting the endpoint have been a good enough “workaround” for now.

Having an external “controller” do have some benefits since it can check that the endpoint is truly stopped and kill it after waiting X seconds etc.

Just let us know if you run into any issues with the controller approach.

Cheers,

Andreas


(Alexander) #18

Andreas, can you elaborate on “check that the endpoint is truly stopped and kill it after waiting X seconds etc”. What to watch out for, what to check and after how long, what do you mean kill it - kill process? What I’m looking for is issues you’ve experienced with this in the past so that we can code to overcome them before we go live.
Thanks


(Andreas Öhlund) #19

Just the fact that most transports will allow inflight messages to complete before shutting down so that can take some time. And yes I meant to kill the process if its still running after {some period of time}