Some of our failed events look like this in ServicePulse:
They appear to be coming from ServiceControl instead of the endpoint that published them. Is this normal?
Hi Matt!
That event (the NServiceBus.EnclosedMessageTypes header will tell you exactly which one) seems to be one of the integration events that is published by ServiceControl, see Using ServiceControl Events • ServiceControl Contracts • Particular Docs for more details.
It seems like the endpoint getting those events fails to process them for some reason, can you share the full stack trace of the error?
Cheers,
Andreas
Andreas,
Thanks for getting back to me. The full stack trace is:
XXX.Util.Capacitor.Exceptions.HttpStatusException: HTTP status code BadRequest indicates failure. Response message: BadRequest. Content: {“object”:null,“validationIssues”:null,“wasSuccessful”:false,“message”:“Invalid Meter Installation Status”}
at XXX.Util.Capacitor.ClientBase.CheckStatusCode(IRestResponse1 response) at XXX.Util.Capacitor.ClientBase.ProcessResponse(IRestResponse1 response, Boolean throwIfContentEmpty)
at XXX.Util.Capacitor.ClientBase.PostAsync(String url, Object request)
at XXX.Capacitor.CustomerConstruction.Endpoint.Clients.TransistorCustomerOperationsConstructionClient.ConstructionActivityCreatedAsync(String jobNumber) in C:\VSTS\VSTSAgents\BuildAgent01_work\2511\s\XXX.Capacitor.CustomerConstruction.Endpoint\Clients\TransistorCustomerOperationsConstructionClient.cs:line 23
at XXX.Capacitor.CustomerConstruction.Endpoint.MessageHandlers.ConstructionActivityHandler.Handle(ConstructionActivityCreated message, IMessageHandlerContext context) in C:\VSTS\VSTSAgents\BuildAgent01_work\2511\s\XXX.Capacitor.CustomerConstruction.Endpoint\MessageHandlers\ConstructionActivityHandler.cs:line 28
at NServiceBus.InvokeHandlerTerminator.Terminate(IInvokeHandlerContext context) in //src/NServiceBus.Core/Pipeline/Incoming/InvokeHandlerTerminator.cs:line 35
at XXX.Util.Capacitor.Behaviors.IncomingLoggingBehavior.Invoke(IInvokeHandlerContext context, Func1 next) at NServiceBus.LoadHandlersConnector.Invoke(IIncomingLogicalMessageContext context, Func2 stage) in //src/NServiceBus.Core/Pipeline/Incoming/LoadHandlersConnector.cs:line 42
at NServiceBus.ScheduledTaskHandlingBehavior.Invoke(IIncomingLogicalMessageContext context, Func2 next) in /_/src/NServiceBus.Core/Scheduling/ScheduledTaskHandlingBehavior.cs:line 23 at NServiceBus.DeserializeMessageConnector.Invoke(IIncomingPhysicalMessageContext context, Func2 stage) in //src/NServiceBus.Core/Pipeline/Incoming/DeserializeMessageConnector.cs:line 34
at ReceivePerformanceDiagnosticsBehavior.Invoke(IIncomingPhysicalMessageContext context, Func2 next) in D:\a\NServiceBus.Metrics\NServiceBus.Metrics\src\NServiceBus.Metrics\ProbeBuilders\ReceivePerformanceDiagnosticsBehavior.cs:line 27 at XXX.Util.Capacitor.Behaviors.TraceIncomingMessagesBehavior.Invoke(IIncomingPhysicalMessageContext context, Func1 next)
at NServiceBus.InvokeAuditPipelineBehavior.Invoke(IIncomingPhysicalMessageContext context, Func2 next) in /_/src/NServiceBus.Core/Audit/InvokeAuditPipelineBehavior.cs:line 27 at NServiceBus.ProcessingStatisticsBehavior.Invoke(IIncomingPhysicalMessageContext context, Func2 next) in //src/NServiceBus.Core/Performance/Statistics/ProcessingStatisticsBehavior.cs:line 32
at NServiceBus.TransportReceiveToPhysicalMessageConnector.Invoke(ITransportReceiveContext context, Func2 next) in /_/src/NServiceBus.Core/Pipeline/Incoming/TransportReceiveToPhysicalMessageConnector.cs:line 61 at NServiceBus.RetryAcknowledgementBehavior.Invoke(ITransportReceiveContext context, Func2 next) in //src/NServiceBus.Core/ServicePlatform/Retries/RetryAcknowledgementBehavior.cs:line 46
at NServiceBus.MainPipelineExecutor.Invoke(MessageContext messageContext) in //src/NServiceBus.Core/Pipeline/MainPipelineExecutor.cs:line 50
at NServiceBus.TransportReceiver.InvokePipeline(MessageContext c) in //src/NServiceBus.Core/Transports/TransportReceiver.cs:line 66
at NServiceBus.TransportReceiver.InvokePipeline(MessageContext c) in //src/NServiceBus.Core/Transports/TransportReceiver.cs:line 66
at NServiceBus.Transport.AzureServiceBus.MessagePump.ReceiveMessage(CancellationToken messageReceivingCancellationToken) in /_/src/Transport/Receiving/MessagePump.cs:line 294
It appears to me that we are in one of our endpoints processing an event and it fails.
We are running an endpoint that receives messages from ServiceControl but this is not that endpoint.
-Mark
Something looks off, based on the stack trace
XXX.Capacitor.CustomerConstruction.Endpoint.MessageHandlers.ConstructionActivityHandler.Handle(ConstructionActivityCreated message, IMessageHandlerContext context)
it seems like the message that is failing is ConstructionActivityCreated but based on your screenshot I can see that the message type shown in ServicePulse is ServiceControl.Contracts.MessageFailed which doesn’t make sense.
Can you confirm that the stack trace and the screenshot come from the same failed message in ServicePulse?
Is that a message that you have retried via ServicePulse?