Outbox in an ASP.NET Core scenario

nservicebus
azureservicebus
(Jimmy Bogard) #1

Let’s suppose we’ve got a web API using ASP.NET Core. As part of an API request, we want to write some data and send a message. However, we’re in a situation where we can’t coordinate the transaction of business data and messages, using something like Azure Service Bus or RabbitMQ.

The outbox right now only works inside of a handler, so how do we use the outbox when we’re not in the context of a handler? In this case, we’re not consuming anything off the outbox in our web API request, but we are producing messages.

Is this possible today?

(Mauro Servienti) #2

Hi @jbogard,

that’s something not doable with Outbox at the moment. We are investigating option in the web reliability space, though.

At the moment your best bet is:

  1. Handle the incoming HTTP request
  2. Send a message to yourself via SendLocal
  3. In the SendLocal handling
    1. store the data
    2. send the outgoing message(s)

(3) will be Outbox enabled. Obviously if the client is waiting for an HTTP response that depends on the data stored at (3.1) the mentioned approach doesn’t work as the incoming HTTP request is already gone, and the only option is to revert back to something like SignalR, that indeed complicates things.

.m

(Jimmy Bogard) #3

Ah, which a client waiting on some kind of response is going to be the vast majority of the cases. If you’re focusing exclusively on ASP.NET Core, that should make your life a lot easier. It’s far, far easier to add middleware with something like services.AddNServiceBus

(Andreas Öhlund) #4

If the client needs a response you might be able to use the callbacks to make all this happen behind the scenes. (web server scale out will be tricky tho)

https://docs.particular.net/nservicebus/messaging/callbacks#when-to-use-callbacks

Ie offloading to messaging and save on some web server threads if things take some time

(Jimmy Bogard) #5

If I’m doing something like the claim check pattern though, I absolutely need that database transaction to happen as part of the request. I don’t really want to shim in async messaging just to use the outbox.

I have a solution I’ve used in MongoDB and SQL, but it’s really not that pretty as I have to create my own messages in my own outbox then “republish” in a background process.

Probably a separate thing, but y’all really should have much deeper integration/support for ASP.NET Core :stuck_out_tongue_winking_eye:

(Szymon Pobiega) #6

Hi @jbogard

What I used to recommend in the past is using SQL Server transport in the web tier (on the boundary of the system) and some other transport, like RabbitMQ inside the system. This allows me to use the same SQL transaction to store data and send messages.

To make this approach work I recommend using NServiceBus.Router to move messages between the transports in a transparent way. The Router supports all types of communication (send, reply, publish).

If you look at this patter from outside, it is just the Outbox but implemented in a distributed way:

  • The web endpoint stores the outbox entries (SQL Server transport messages)
  • The Router endpoint removes the outbox entries and dispatches “real” messages

There is a sample that demonstrates how to use this approach to do atomic update-and-publish in the web controller.

Szymon

(Jimmy Bogard) #7

Ahhhhh that makes sense! I like this approach.

(Szymon Pobiega) #8

@jbogard I tried to simplify that a bit and created NSeviceBus.Connector.SqlServer (source here) which allows you to easy inject into a controller an instance of IMessageSession that shares connection and transaction with data access library.

The controller code is not aware of the connector. It just uses the session:

public SendMessageController(IMessageSession messageSession, SqlConnection conn, SqlTransaction trans)
{
    this.messageSession = messageSession;
    this.conn = conn;
    this.trans = trans;
}

[HttpGet]
public async Task<string> Get()
{
    await messageSession.Send(new MyMessage())
        .ConfigureAwait(false);

    await messageSession.Publish(new MyEvent())
        .ConfigureAwait(false);

    using (var command = new SqlCommand("insert into Widgets default values;", conn, trans))
    {
        await command.ExecuteNonQueryAsync()
            .ConfigureAwait(false);
    }

    return "Message sent to endpoint";
}

In the setup code:

//Connects to MSMQ transport used by other endpoints
var connectorConfig = new ConnectorConfiguration<MsmqTransport>(
    name: "WebApplication",
    sqlConnectionString: ConnectionString,
    customizeConnectedTransport: extensions => {},
    customizeConnectedInterface: configuration =>
    {
        //Required because connected transport (MSMQ) does not support pub/sub
        configuration.EnableMessageDrivenPublishSubscribe(storage);
    });

//Routing for commands
connectorConfig.RouteToEndpoint(typeof(MyMessage), "Samples.ASPNETCore.Endpoint");

//Start the connector
connector = connectorConfig.CreateConnector();
await connector.Start();

//Register per-request SQL Server connection
services.UseSqlServer(ConnectionString);
//Register automatic opening of per-request transaction
services.UseOneTransactionPerHttpCall();
//Register per-request connector-based message session based on connection/transaction
services.UseNServiceBusConnector(connector);

There is a sample included in the repo that I intend to eventually move to NServiceBus doco site. My goal is to package the router and the send-only endpoint in such a way that for the user it looks just like an ordinary NServiceBus send-only endpoint. The only difference is the fact that it requires two queues – one in the SQL Server and the other in the external transport. That second queue is used for managing pub/sub in case the external transport does not handle it natively.

Szymon

(Jimmy Bogard) #9

Ah, that’s really nice, thanks! I’ll give it a go.