Hi,
I’ve been using NServicebus 7.8.4 in a production environment for about 2 years, and every now and then we notice “lost” published messages.
Our logic goes like this:
public async Task Handle(UpdateDetected event, IMessageHandlerContext context)
{
// check if this event ever started a SAGA
var eventAlreadyProcessed =await IsEventInTablestorage (event.Key).ConfigureAwait(false);
if(eventAlreadyProcessed)
{
return;
}
switch (SagaToStart)
{
case string s when s.Equals(nameof(event), StringComparison.OrdinalIgnoreCase):
await context.Publish<event>(msg =>
{
msg.Key = event.Key;
msg.X = event.X;
msg.Y = event.Y;
}).ConfigureAwait(false);
break;
case: // other cases...
case: // other cases...
default:
throw Exception("unknown SAGA/wrong configuration");
}
// mark that this event started a SAGA
await AddEventToTableStorage (event.Key).ConfigureAwait(false);
}
this works great most of the time, approximately 1 in 2000 messages get lost this way.
I see in my logging that the context.Publish is being called with the correct configuration & after that the AddEventToTableStorage, but no sign of a started SAGA or an error in Servicepulse, ServiceInsight, …
The problem does not occur with any specific type of SAGA, it seems the publish returns correctly but the message is never handed over to Azure Service bus.
Is there anything that I can check or enable to troubleshoot this further?