Shutdown of Services takes a long time with ServiceControl Metrics plugin enabled

After enabling the Metrics plugin in the endpoints, they take around the time of ‘MetricsInterval’ until the shutdown is complete. The endpoints are running as windows services.
Configuration is as follows.
var metrics = endpointConfiguration.EnableMetrics();
metrics.SendMetricDataToServiceControl(“Monitoring”, configuration.MetricsInterval);

When I set MetricsInterval to 10 minutes, shutdown takes around this time. When I set it to 1 second, the shutdown is more or less immediately.

Is this the expected behavior? Interestingly the slow shutdown doesn’t happen when running the process in console (for graceful shutdown, not killing the process).

Expected production settings for MetricsInterval would be expected to be around 30" to 1’.
This would slow down the deployment process significantly for us

How are you currently hosting your endpoints?

Hi Philipp,

Looks like we have a bug

I’ve raised Shutdown of endpoint can hang until ServiceControlReportingInterval is reached · Issue #61 · Particular/NServiceBus.Metrics.ServiceControl · GitHub. We will triage this and report back to you soon

Regards
Daniel

1 Like

Host.Start basically does
this.endpoint = await Endpoint.Start(endpointConfiguration);

and Host.Stop
await this.endpoint.Stop();

public static class Program
{
    public static async Task Main(string[] args)
    {
        string absoluteFolderPathOfExecutable = GetAbsoluteFolderPathOfExecutable();
        Directory.SetCurrentDirectory(absoluteFolderPathOfExecutable);

        log4net.Config.XmlConfigurator.ConfigureAndWatch(
            new FileInfo(Path.Combine(absoluteFolderPathOfExecutable, "log4net.config")));

        LogManager.Use<Log4NetFactory>();
        var host = new Host();

        if (args.Contains("--run-as-service"))
        {
            RunAsService(host);
            return;
        }

        await RunInConsole(host);
    }

    // ReSharper disable once PossibleNullReferenceException
    private static string GetAbsoluteFolderPathOfExecutable() =>
        Path.GetDirectoryName(Assembly.GetEntryAssembly().Location);

    private static void RunAsService(Host host)
    {
        using (var windowsService = new WindowsService(host))
        {
            ServiceBase.Run(windowsService);
        }

        log4net.LogManager.Shutdown();
    }

    private static async Task RunInConsole(Host host)
    {
        Console.Title = host.EndpointName;

        var tcs = new TaskCompletionSource<object>();
        Console.CancelKeyPress += (sender, e) =>
        {
            e.Cancel = true;
            tcs.SetResult(null);
        };

        await host.Start();
        await Console.Out.WriteLineAsync("Press Ctrl+C to exit...");

        await tcs.Task;
        await host.Stop();

        log4net.LogManager.Shutdown();
    }
}
   
public class WindowsService : ServiceBase
{
    private readonly Host host;

    public WindowsService(Host host) => this.host = host;

    protected override void OnStart(string[] args) => this.host.Start().GetAwaiter().GetResult();

    protected override void OnStop() => this.host.Stop().GetAwaiter().GetResult();
}

Hi Philipp

We released a bug fix

https://discuss.particular.net/t/nservicebus-metrics-servicecontrol-3-0-3-and-2-1-2-patch-releases-available

Hope that helps

Regards
Daniel

1 Like