Messaging Bridge "Install" mode

Architecture & Design
,
1

I’m using the Messaging Bridge and getting to grips with it slowly. I have a scenario where I want it to run as part of a deployment pipeline to create it’s MSMQ queues and Azure Service Bus resources. This is because the deployment agent has permissions to do such things, whereas when running in it’s “regular” mode in a Container, it’s not permitted.

I want to support a “–run-installers” command-line argument, but at the moment the only control I seem to have is to set “CreateQueues” and “AutoCreateQueues” (on the MsmqTransport/AzureServiceBusTransport and BridgeTransport respectively). There is no equivalent (that I can see) of the EndpointConfiguration.EnableInstallers().

I have tried using things like IHost.RunAsync(new CancellationTokenSource(TimeSpan.FromSeconds(45).Token); but I get weird exceptions when it tries to shut down. I’ve tried higher and lower values for the timeout, to no avail.

I’d like to know the idiomatic approach to this.

Example exception:
Unhandled exception. System.TimeoutException: The operation did not complete within the allocated time 00:01:00 for object drain.
at Microsoft.Azure.Amqp.AsyncResult.End[TAsyncResult](IAsyncResult result)
at Microsoft.Azure.Amqp.ReceivingAmqpLink.DrainAsyncResult.End(IAsyncResult result)
at Microsoft.Azure.Amqp.ReceivingAmqpLink.<>c.b__35_1(IAsyncResult r)
at System.Threading.Tasks.TaskFactory1.FromAsyncCoreLogic(IAsyncResult iar, Func2 endFunction, Action1 endAction, Task1 promise, Boolean requiresSynchronization)
— End of stack trace from previous location —
at Azure.Messaging.ServiceBus.Amqp.AmqpReceiver.CloseAsync(CancellationToken cancellationToken)
at Azure.Messaging.ServiceBus.ServiceBusReceiver.CloseAsync(CancellationToken cancellationToken)
at Azure.Messaging.ServiceBus.ServiceBusReceiver.DisposeAsync()
at Azure.Messaging.ServiceBus.ReceiverManager.CloseReceiverIfNeeded(CancellationToken cancellationToken)
at Azure.Messaging.ServiceBus.ServiceBusProcessor.CloseAsync(CancellationToken cancellationToken)
at NServiceBus.Transport.AzureServiceBus.MessagePump.StopReceive(CancellationToken cancellationToken) in //src/Transport/Receiving/MessagePump.cs:line 223
at NServiceBus.Transport.AzureServiceBus.MessagePump.StopReceive(CancellationToken cancellationToken) in //src/Transport/Receiving/MessagePump.cs:line 238
at NServiceBus.Raw.RawTransportReceiver.Stop(CancellationToken cancellationToken) in //src/NServiceBus.MessagingBridge/RawEndpoints/RawTransportReceiver.cs:line 62
at NServiceBus.Raw.RunningRawEndpointInstance.StopReceiving(CancellationToken cancellationToken) in //src/NServiceBus.MessagingBridge/RawEndpoints/RunningRawEndpointInstance.cs:line 30
at NServiceBus.Raw.RunningRawEndpointInstance.Stop(CancellationToken cancellationToken) in //src/NServiceBus.MessagingBridge/RawEndpoints/RunningRawEndpointInstance.cs:line 44
at RunningBridge.Stop(CancellationToken cancellationToken) in //src/NServiceBus.MessagingBridge/RunningBridge.cs:line 15
at Microsoft.Extensions.Hosting.Internal.Host.ForeachService[T](IEnumerable1 services, CancellationToken token, Boolean concurrent, Boolean abortOnFirstException, List1 exceptions, Func`3 operation)
at Microsoft.Extensions.Hosting.Internal.Host.StopAsync(CancellationToken cancellationToken)
at Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.WaitForShutdownAsync(IHost host, CancellationToken token)
at Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.RunAsync(IHost host, CancellationToken token)
at Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.RunAsync(IHost host, CancellationToken token)
at Program.$(String args) in D:\a\1\s\src\Thin.Esb.Bridge\Program.cs:line 150
at Program.(String args)

2

Hi Neil,

Can you please clarify which queues you’re trying to get the Bridge to create?

Is it ONLY the proxy queues? Or are you trying to get it to create the MSMQ queues or the Azure Service Bus queues?

3

Ahh a very good point. Yes, the proxy queues. There’s a chicken-and-egg situation going on. I can’t run the bridge before an MSMQ publisher because the MSMQ publisher’s queue doesn’t exist to send subscription ControlData messages to. But if the Bridge hasn’t run, it doesn’t create the MSMQ proxy queues for MSMQ subscribers to send their subscription ControlData messages to.

Now, for MSMQ, I could add something to the deployment process that creates these queues in advance easily enough. However, for the Azure Service Bus resources I think that’s more risky. As we know the transition from v4 to v5 of the ASB library has the topic name breaking change, and so I’m keen to let the Bridge do it’s thing rather than trying to reverse-engineer it and build something myself. I’d like the bridge to run, create whatever it needs, wherever it needs, and then quit. When run “properly” in the Container App, all that stuff will be there so it can just fire up and continue running to do it’s thing.

4

The idea is you would have the MSMQ system already running in production.

This would mean that the MSMQ queues are already created by the time the bridge runs. The Bridge would then run, connect to MSMQ and create the proxy queues that are on the Azure Service Bus side. At this stage it would also send the subscription messages to MSMQ.

The ASB side would do the same, but in reverse. It would connect to the ASB broker and create the corresponding MSMQ proxy queues.

TL;DR: The bridge expects the underlying queues to already be in place, and it will only create the proxy queues.

Which means for your deployment process, you’d want:

  1. The MSMQ endpoints to run using .EnableInstallers();
  2. Shut down the MSMQ endpoints
  3. Run the ASB endpoints using .EnableInstallers(); or Installer.Setup(endpointConfiguration);
  4. Shut down the ASB endpoints
  5. Run the bridge with AutoCreateQueues set to true on both sides of the configuration
5

Ah so the idea is to run a service with installers, then start the bridge, then start the service. That’s what’s tripping me up, I think. Our deployment pipeline runs the service with installers, then starts it. A separate pipeline deploys the bridge. So if we’re deploying a brand new service for the first time, that’s when it’ll trip us up (because it’s resources don’t exist for the bridge to work with, and it can’t be started until the bridge has run and created proxy resources).

Either way though, the bridge needs to run once with installers so it creates it’s proxy resources with the permissions of the deployment agent, and then when it runs normally, those resources are already there (because it’s not allowed to create them when running normally).

There doesn’t seem to be an idiomatic way for the bridge to run, creating it’s proxy resources, then shut down automatically.

6

Correct.

The Bridge does not have a “Run Installers Only” mode.

7

Fair enough. In the spirit of following the principle of least privilege, is there an official recommended idiomatic way to create a Bridge’s proxy resources? I’m trying to follow the advice from Installers • NServiceBus • Particular Docs

8

The same as for running installers during endpoint startup: Start the Bridge with autocreate when an install flag is present, then stop the bridge once it’s running.

If the install flag isn’t present, run it without AutoCreate.

Something like this as a quick sample should work

        var runInstallers = Environment.GetCommandLineArgs().Any(x => string.Equals(x, "/runInstallers", StringComparison.OrdinalIgnoreCase));

        var builder = Host.CreateDefaultBuilder();
        var host = builder
            .UseNServiceBusBridge((ctx, bridgeConfiguration) =>
            {
                var asbBridgeEndpoint = new BridgeEndpoint("Samples.MessagingBridge.AsbEndpoint");
                asbBridgeEndpoint.RegisterPublisher<MyEvent>("Samples.MessagingBridge.MsmqEndpoint");

                var asbBridgeTransport = new BridgeTransport(new AzureServiceBusTransport(connectionString));
                asbBridgeTransport.AutoCreateQueues = runInstallers;
                asbBridgeTransport.HasEndpoint(asbBridgeEndpoint);
                bridgeConfiguration.AddTransport(asbBridgeTransport);

                var msmqBridgeEndpoint = new BridgeEndpoint("Samples.MessagingBridge.MsmqEndpoint");

                msmqBridgeEndpoint.RegisterPublisher<OtherEvent>("Samples.MessagingBridge.AsbEndpoint");

                var msmqBridgeTransport = new BridgeTransport(new MsmqTransport());
                msmqBridgeTransport.AutoCreateQueues = runInstallers;
                msmqBridgeTransport.HasEndpoint(msmqBridgeEndpoint);
                bridgeConfiguration.AddTransport(msmqBridgeTransport);
            })
            .Build();

        if (runInstallers)
        {
            await host.StartAsync();
            await host.StopAsync();
        }
        else
        {
            await host.RunAsync();
        }
9

I think perhaps this is the magic sauce I had not appreciated - Start/Stop vs. Run. Thanks for the pointer.

1 Like