Exception thrown when sharing ISqlStorageSession-transaction with DbContext

[DevJ]

Hello,

We are looking into the samples on setting up Outbox with EF Core. Our situation is slightly different in the sense that we prefer using the AddDbContext-method from EF Core. In addition, the creation of the context is not in our solution but in an external package. Which is why we devised a setup like this:

public ContextWrapper RegisterWrapperScoped(IServiceProvider serviceprovider)
{
   var session =  serviceprovider.GetRequiredService<ISqlStorageSession>();
   var context =  serviceprovider.GetRequiredService<ReceiverDataContext>();
  
   // context.Database.SetDbConnection(session.Connection); - possible fix
   context.Database.UseTransaction(session.Transaction);

   session.OnSaveChanges((s, token) => context.SaveChangesAsync(token));
   return new ContextWrapper(context);
}

This method would be called when creating our IServiceCollection container (we are using the externally managed mode).

Problem:
Our problem is that we are getting an exception when calling this line:
context.Database.UseTransaction(session.Transaction);

This line causes the following exception:

System.InvalidOperationException: ‘The specified transaction is not associated with the current connection. Only transactions associated with the current connection may be used.’

We know that we can resolve this problem by calling the following before .UseTransaction:

context.Database.SetDbConnection(session .Connection);

However, this does raise doubts on the cleanliness of the code. Are there any samples (e.g. by Particular) on how we can properly have the DbContext use the same transaction as the ISqlStorageSession when using the AddDbContext-method from Microsoft’s EF Core?

Additional info on the exception

• context.Database.GetConnectionString() == session .Connection.ConnectionString is true
• context.Database.GetDbConnection() == session .Connection is false
• session.Connection == sqlStorageSession.Transaction.Connection is true
• the objectId-property in the SqlConnections differ between the session and the dbContext, so it appears that we may need to use the same connection when initializing the session and context.
• we are using EntityFramework Core (version 6)

If there are any suggestions or questions, we’d love to hear it!

[danielmarbach]

Hi @DevJ

Thanks for reaching out to us through the discuss forum.

In addition, the creation of the context is not in our solution but in an external package

Could you elaborate this a bit? How is that context provided? Is this external package using some sort of hook into the service collection and registering it scoped? Are does the package simply provide the ReceiverDataContext and you have to register it using the AddDbContext-method from EF Core and that is your preferred way of registering things?

Would you also explain a bit more the ContextWrapper? I don’t understand why that is needed.

No going back to the registration method. My understanding is that AddDbContext is a convenience method that provides certain overloads for commonly used entity framework context registration scenarios. For anything that falls outside that using custom registrations is totally acceptable. In the case with NServiceBus with SQL Persistence (or even combined with the SQL transport) NServiceBus manages the connection and transaction. These “connection means” need to be hooked up with the entity framework context that is created. For this scenario a custom registration approach of the context is totally acceptable

It is not enough to register “just the transaction” because then the context would manage potentially its own connection which the transaction doesn’t belong to leading the the problem you described.

we may need to use the same connection when initializing the session and context.

Yes you have to.

When it comes to the registrastration I do not know whether the context is only ever used within message handler scenarios. If it isn’t a more robust way of writing the code above would be something like the following

public ContextWrapper RegisterWrapperScoped(IServiceProvider serviceprovider)
{
   var context =  serviceprovider.GetRequiredService<ReceiverDataContext>();
   if (serviceprovider.GetService<ISynchronizedStorageSession>() is ISqlStorageSession { Connection: not null } session)
   {
     context.Database.SetDbConnection(session.Connection);
     context.Database.UseTransaction(session.Transaction);
     session.OnSaveChanges((s, token) => context.SaveChangesAsync(token));
   }
   
   return new ContextWrapper(context);
}

but I would entirely move this code into a custom service collection extension like the following

services.AddDbContext<ReceiverDataContext>(ServiceLifetime.Scoped);
services.AddScoped(provider =>
{
    var context = provider.GetRequiredService<ReceiverDataContext>();
    if (provider.GetService<ISynchronizedStorageSession>() is ISqlStorageSession { Connection: not null } session)
    {
        context.Database.SetDbConnection(session.Connection);
        context.Database.UseTransaction(session.Transaction);
        session.OnSaveChanges((s, token) => context.SaveChangesAsync(token));
    }

    return new ContextWrapper(context);
});

However, this does raise doubts on the cleanliness of the code.

Can you elaborate on that one? What is bothering you there? What specific functionality do you need from the AddDbContext convenience method that you feel like would be missing here? Maybe we can give you concrete guidance on how to achieve that.

Regards,
Daniel

[DevJ]

Hello,

Thanks for the reply.

How is that context provided?

The external package provides a method that takes an IServiceCollection-object and adds a DbContext registration by calling AddDbContext().

What specific functionality do you need

Because AddDbContext isn’t called in our code, we needed a way to call the aforementioned hook-up code to connect the DbContext with NServiceBus’ connection means.

The ContextWrapper is introduced to have the hook-up code called when a class needs access to the DbContext. Dependent classes would then resolve the ContextWrapper instead of the DbContext, which leads to the SetDbConnection and UseTransaction being called.

Is there is an alternative way to always call the hook-up logic when resolving the DbContext for a dependent class (e.g. a message handler)? Preferably without changing the external package’s registration of the DbContext in the DI container.

Some more context on our use case:
Our use case is that our code currently doesn’t use the SynchronizedStorageSession API to store our application data. This was because NServiceBus was introduced into our systems at a later time. Instead, we have some data layer frameworks that start and commit their own transactions (e.g. ADO and EF Core).

With Outbox, it appears that our systems are required to persist application + outbox data through the transaction in the SynchronizedStorageSession. If my understanding is correct, we need to refactor our systems to access the connection/transaction objects in the handler code and pass that to our data layer. The layer should then not commit the transaction (if inside the scope of a handler) and leave the committing to NServiceBus.

This would require some major refactoring. Instead, is there a code pattern / API to have NServiceBus open a SQL transaction and automatically cause all subsequent database transactions to become nested inside this all-encompassing ambient transaction (perhaps the same as the SynchronizedStorageSession)?

[danielmarbach]

Hi DevJ

@DevJ

Because AddDbContext isn’t called in our code, we needed a way to call the aforementioned hook-up code to connect the DbContext with NServiceBus’ connection means.

I might still not see / understand things clearly. Sorry if I keep on asking questions here. ServiceCollection ServiceDescriptor registrations are ordered. So that means somehow there is a hook that wires up your code to call that code that then does AddDbContext.

So either their code is executed first, which means you can call your code and override theirs by getting the right ServiceDescriptors out of the collection or overriding the ReceiverDataContext registration entirely. Or your code is executed first which means you can register the ReceiverDataContext yourself and the subsequent AddDbContext call of other code would do does a TryAdd calls which are effectively NoOps because you already added the corresponding registrations.

Is there is an alternative way to always call the hook-up logic when resolving the DbContext for a dependent class (e.g. a message handler)?

If I understand you correctly, the most preferred approach for you would be to directly use the ReceiverDataContext and not having to wrap things, right? I think once we have gone through the above questions and comments we can go back to this.

we have some data layer frameworks that start and commit their own transactions (e.g. ADO and EF Core)

Yeah that is problematic.

With Outbox, it appears that our systems are required to persist application + outbox data through the transaction in the SynchronizedStorageSession. If my understanding is correct, we need to refactor our systems to access the connection/transaction objects in the handler code and pass that to our data layer.

The whole idea of wiring things up via DI for example to the entity framework context is that your code that directly operates on the context doesn’t have to change or only requires minimal changes. The ISqlStorageSession interface is also available as a scoped dependency and can be used to hook up things to your other Ado.NET layer as necessary.

This would require some major refactoring. Instead, is there a code pattern / API to have NServiceBus open a SQL transaction and automatically cause all subsequent database transactions to become nested inside this all-encompassing ambient transaction

That’s a tricky one. You might be able to set a TransactionScope in some places that then automatically “flows” across your async code. Entity framework also has the concept of interceptors that might help. To register those, you have to have access to the DbContextOptions, though. I think we would need to look into your code and abstractions to be able to figure out a suitable solution. If you reach out to support, we can hook you up with the account manager to figure out those details.

Regards,
Daniel

[DevJ]

Hello,

So that means somehow there is a hook that wires up your code to call that code that then does AddDbContext .

Correct, the flow looks somewhat like this:

public void ConfigureServices(IServiceCollection services)
{
//...
   services.ExternalAddDbContextCall(); // this calls AddDbContext in the external code

   PerformHookUpCode(services);
//...
}

private void PerformHookUpCode(IServiceCollection services)
{
   // Here we need to write code such that 
   // `ReceiverDataContext.Database.UseTransaction(session.Transaction);` is called.
   // when ReceiverDataContext is resolved, so the context uses the session transaction.
}

Of course, we could try to replace the registation of ReceiverDataContext in the DI container instead of using a ContextWrapper. We tried to avoid it because the registration logic of the ReceiverDataContext could change after a package update. This could potentially lead to us replacing the external registration with a faulty one.

If you reach out to support, we can hook you up with the account manager to figure out those details.

Yes, will do. We are currently assessing in what ways our systems handle db transactions, including the situation above. Afterwards, we’ll get in touch with support to figure out our options.

[danielmarbach]

Hi @DevJ

Given that you will be using the ReceiverDataContext in the code that expects it to be wired up with NServiceBus why not doing a keyed registration?

            c.AddKeyedScoped("WiredUpWithUnitOfWork",static (provider, _) =>
            {
                var context = provider.GetRequiredService<ReceiverDataContext>();
                if (provider.GetService<ISynchronizedStorageSession>() is ISqlStorageSession { Connection: not null } session)
                {
                    context.Database.SetDbConnection(session.Connection);
                    context.Database.UseTransaction(session.Transaction);
                    session.OnSaveChanges((s, token) => context.SaveChangesAsync(token));
                }
                return context;
            });

and then in your handler code

public class OrderAcceptedHandler : IHandleMessages<OrderAccepted>
{
    public OrderAcceptedHandler([FromKeyedServices("WiredUpWithUnitOfWork")] ReceiverDataContext dataContext)
    {

    }

    public Task Handle(OrderAccepted message, IMessageHandlerContext context)
    {
        
        return Task.CompletedTask;
    }
}

No need to decorate, and also no need to override existing registrations. If you prefer to flip around and make the external one the keyed one, you could do the following (there might be one or two more edge cases my pseudo code completely blinds out):

            var registration = services.Single(x => x.ServiceType == typeof(ReceiverDataContext));
            services.Remove(registration);
            services.Add(new ServiceDescriptor(registration.ServiceType, "External", registration.ImplementationType ?? registration.ServiceType, registration.Lifetime));

            services.AddScoped(static (provider) =>
            {
                var context = provider.GetRequiredKeyedService<ReceiverDataContext>("External");
                if (provider.GetService<ISynchronizedStorageSession>() is ISqlStorageSession { Connection: not null } session)
                {
                    context.Database.SetDbConnection(session.Connection);
                    context.Database.UseTransaction(session.Transaction);
                    session.OnSaveChanges((s, token) => context.SaveChangesAsync(token));
                }
                return context;
            });

Regards,
Daniel