Saga Data issue with NSB 7.2 and Azure Storage Table

Hello!
We are experiencing issues with Saga correlation property. Any help to get us to the right direction would be much appreciated!

Thank you,
Kebin

Issue:

We are in the transition of upgrading one of our endpoints. After the upgrade, when a new Saga is to be created, and setting the correlation property to value such as “Users/123” we are getting storage exception.

Current configuration details:

  • netcoreapp 3.1

  • NSB 7.2

  • NServiceBus.Azure.Transports.WindowsAzureStorageQueues 8.1.4

  • NServiceBus.Newtonsoft.Json 2.2.0

Configuration:

endpointConfiguration.SendFailedMessagesTo(“error”);
endpointConfiguration.UseSerialization();
endpointConfiguration.EnableInstallers();

var transport = endpointConfiguration
.UseTransport()
.ConnectionString(azureConnectionString);

endpointConfiguration
.UsePersistence()
.ConnectionString(azureConnectionString);

Saga Data class:

public class StudentQuizesSagaData : ContainSagaData
{
public string StudentId { get; set; }
public int OverallRunningTally { get; set; }
public string CurrentIntervalYyyyMmDdInString { get; set; }
}

protected override void ConfigureHowToFindSaga(SagaPropertyMapper mapper)
{
mapper.ConfigureMapping(msg => msg.StudentId).ToSaga(data => data.StudentId);
mapper.ConfigureMapping(msg => msg.StudentId).ToSaga(data => data.StudentId);
mapper.ConfigureMapping(msg => msg.StudentId).ToSaga(data => data.StudentId);
}

Exception

“NServiceBus.ExceptionInfo.StackTrace”:"Microsoft.WindowsAzure.Storage.StorageException: Unexpected response code, Expected:OK or NotFound, Received:BadRequest\n

at Microsoft.WindowsAzure.Storage.Core.Executor.Executor.ExecuteAsyncInternal[T](RESTCommand`1 cmd, IRetryPolicy policy, OperationContext operationContext, CancellationToken token)\n

at NServiceBus.Persistence.AzureStorage.SecondaryIndices.SecondaryIndexPersister.FindSagaIdAndCreateIndexEntryIfNotFound[TSagaData](String propertyName, Object propertyValue)\n

at NServiceBus.Persistence.AzureStorage.AzureSagaPersister.GetByCorrelationProperty[TSagaData](String propertyName, Object propertyValue, SynchronizedStorageSession session, ContextBag context, Boolean triedAlreadyOnce)\n

at NServiceBus.PropertySagaFinder`1.Find(IBuilder builder, SagaFinderDefinition finderDefinition, SynchronizedStorageSession storageSession, ContextBag context, Object message)\n

at NServiceBus.SagaPersistenceBehavior.Invoke(IInvokeHandlerContext context, Func`2 next)\n

at NServiceBus.LoadHandlersConnector.Invoke(IIncomingLogicalMessageContext context, Func`2 stage)\n

at NServiceBus.ScheduledTaskHandlingBehavior.Invoke(IIncomingLogicalMessageContext context, Func`2 next)\n

at NServiceBus.InvokeSagaNotFoundBehavior.Invoke(IIncomingLogicalMessageContext context, Func`2 next)\n

at NServiceBus.DeserializeMessageConnector.Invoke(IIncomingPhysicalMessageContext context, Func`2 stage)\n

at NServiceBus.SubscriptionReceiverBehavior.Invoke(IIncomingPhysicalMessageContext context, Func`2 next)\n

at NServiceBus.UnitOfWorkBehavior.InvokeUnitsOfWork(IIncomingPhysicalMessageContext context, Func`2 next)\n

at NServiceBus.UnitOfWorkBehavior.InvokeUnitsOfWork(IIncomingPhysicalMessageContext context, Func`2 next)\n

at NServiceBus.ProcessingStatisticsBehavior.Invoke(IIncomingPhysicalMessageContext context, Func`2 next)\n

at NServiceBus.TransportReceiveToPhysicalMessageConnector.Invoke(ITransportReceiveContext context, Func`2 next)\n

at NServiceBus.MainPipelineExecutor.Invoke(MessageContext messageContext)\n

at NServiceBus.Transport.AzureStorageQueues.AtLeastOnceReceiveStrategy.Receive(MessageRetrieved retrieved, MessageWrapper message)\n

Request Information\nRequestID:1815c36a-e002-001a-78cc-21aed0000000\n

RequestDate:Sun, 03 May 2020 21:25:44 GMT\n

StatusMessage:Bad Request\nErrorCode:\n

ErrorMessage:Bad Request - Error in query syntax.\n

RequestId:1815c36a-e002-001a-78cc-21aed0000000\n

Time:2020-05-04T04:25:44.8563957Z\n"

Comments:

  • StudentId would have values such as: Users/123, Users/124 etc…
  • If we change from “/” to “_”, everything is good.
  • This used to work in NSB 5.x

Are there guides that we are missing to upgrade or patch previous saga data etc?

Storage Tables partition and row keys do not allow forward slash as a value. See Microsoft documentation here.

Has the previous version of your system replaced it with an underscore?

Hi Sean,
Thanks for the quick response. Yes that makes sense for the new Saga. But what happens to the previous version of the Saga data? How will new incoming message find existing Saga? Do we update the property by replacing all old data from “/” to “_” ?

I’ve attached screenshots of what the new vs. old saga data looks like. New incoming messages create a new Saga.

Kebin,

Are you migrating straight from NServiceBus v5 / NServiceBus.Azure v6 to NServiceBus v7 and NServiceBus.Persistence.AzureStorage v2?

Hi Sean,
Yes from v5 to v7. There’s not much going on in this endpoint and we didn’t encounter any other issues.

Thanks.

Hi Kebin,

I would highly advise to review the migration guides for NServiceBus (Upgrade Guides • NServiceBus • Particular Docs), for the transport (Upgrade Guides • NServiceBus • Particular Docs) and for the persistence (Upgrade Guides • NServiceBus • Particular Docs).

For Azure Storage persistence you’ll see that the upgrade from NServiceBus.Azure version 6 to NServiceBus.Persistence.AzureStorage version 1 had a few prerequisites.

Upgrades from NServiceBus.Azure versions 6.2.3 and below will need to apply the saga de-duplication patch followed by the saga index patch before completing the remainder of these upgrade steps.

Those need to take place first:

  1. Azure Storage Persistence upgrade version 6.2.3 to 6.2.4 • Azure Table Persistence • Particular Docs
  2. Azure Storage Persistence upgrade Version 6.2.4 to 6.2.5 • Azure Table Persistence • Particular Docs

Hi Sean,
Thanks for all the links. I’ve gone through most of the links above. Looks we may not have been affected by “saga duplication bug”, but will go ahead and run the patch anyway.

Will those two patches actually resolve our correlation id issue though? It looks like we’d still need to fix the property by changing from / to _

Kebin,

Will be honest with you - version 5 is a very old version. I don’t recall the persistence accepting forward slash as it was never a valid character. If after migration you’re still having this issue, the best option would be to raise a support case (Support options • Particular Software).

Hi Sean,
Thanks for the suggestions. Will definitely raise a support case if we don’t get this resolved today.

Thanks for chiming in here.
Kebin