Production service stops after 5 seconds


#1

EDIT: Solved. It was a poison message, but for whatever reason it was showing no info like stack track or exception. Just closes which it’s never done before.

I have an NServiceBusHost 3.8 running with MSMQ dumping to SQLServer. The service has been in place and running nearly unchanged since 2013.

This past weekend the service stopped and when starting again, always stops. Running the service host directly to see console output the app starts up normally, processes some message then closes with no errors.

I changed logging to debug and there are no init issues, and no errors.

One odd thing is that it seems to run for almost exactly 5 seconds each time. Processing as many messages as it can, and the process just ends.

Are there any known mechanisms that might cause such behavior? License file is being brought in and active. I don’t know what else might end it automatically each time.

The log cuts off in the middle of processing a message, so it isn’t like it ran out of messages and timed out or something similar (have 600K messages pending at the moment.

Because of the 5 second time, and no changes to the production code or environment, I’m hoping this is gonna ring a bell for someone.


(Bill Matson) #2

Anything interesting in the system event log? Are you running into MSMQ
file size quota issues (especially due to transactional DLQ)?

Bill


(Indu Alagarsamy) #3

Hi,

It was a poison message? If the message could not be processed then it gets forwarded to the error queue. Was there any information in the event log that helped you diagnose this?

Thanks much!
Indu Alagarsamy
Particular Software


#4

Correct, it was expected to do that. It was not moved to the error queue (it is and has been configured previously).

There was no indication in any log (local, app, system, msmq) that any problem occurred, the app just stops.

It was only when I ran it enough times manually to get through the other messages and at the end there continued to be 1 message that would crash the app. It had malformed xml.


(Indu Alagarsamy) #5

Hi,

Thanks for your response.

I believe the latest versions of NServiceBus has appropriate protection against malformed messages. There were several fixes to the XmlSerializer that were done since NServiceBus Version 3. Just to give you an example, issues like
https://github.com/Particular/NServiceBus/issues/934.

We back-port all our important bug fixes to all the supported versions. However, NServiceBus Version 3 is also no longer supported. Our support policy below shows you the currently supported versions:
https://docs.particular.net/nservicebus/upgrades/support-policy

While I am extremely glad to hear that this version has been working for you well, it might be a good idea to upgrade to keep up with the fixes so that you don’t run into any further issues. We have comprehensive upgrade guides that help you with upgrading major versions as well. Don’t hesitate to reach out if you need any assistance in this regard.

Cheers,
Indu Alagarsamy
Particular Software