Using Polly inside of message handler

Architecture & Design
1

We are creating code to call a Rest api to attach a file and upload it to the host. This will be triggered by a message sent and picked up by a standard NSB message handler. We are using RestSharp to do this. We were considering using Polly to smooth out any retries etc with a policy. A colleague warned against this as they said NServiceBus should handle this natively. I have seen conflicting advice online and am tempted to still use Polly. Pros and cons? Here is the current Api code with the Polly policy.

using System;
using System.Configuration;
using System.Linq;
using System.Net;
using System.Net.Http;
using System.Text;
using System.Threading.Tasks;
using Polly;
using RestSharp;

namespace Et.Sample.Console.Rest.Client
{
    public class Post
    {
        private readonly string filePath;
        private readonly string fileType;
        private readonly string host;
        private readonly int maxRetries;
        private readonly string password;
        private readonly string route;
        private readonly string userId;
        
        public Post()
        {
            filePath = ConfigurationManager.AppSettings["filePath"];
            fileType = ConfigurationManager.AppSettings["fileType"];
            host = ConfigurationManager.AppSettings["host"];
            maxRetries = int.Parse(ConfigurationManager.AppSettings["maxretries"]);
            password = ConfigurationManager.AppSettings["password"];
            route = ConfigurationManager.AppSettings["route"];
            userId = ConfigurationManager.AppSettings["userid"];
        }

        public async Task<string> Execute()
        {
            // Handle both exceptions and return values in one policy
            HttpStatusCode[] httpStatusCodesWorthRetrying =
            {
                HttpStatusCode.RequestTimeout, // 408
                HttpStatusCode.InternalServerError, // 500
                HttpStatusCode.BadGateway, // 502
                HttpStatusCode.ServiceUnavailable, // 503
                HttpStatusCode.GatewayTimeout // 504
            };

            var result = await Policy
                .Handle<HttpRequestException>()
                .OrResult<RestResponse>(r => httpStatusCodesWorthRetrying.Contains(r.StatusCode))
                .WaitAndRetryAsync(new[]
                {
                    TimeSpan.FromSeconds(15),
                    TimeSpan.FromSeconds(30),
                    TimeSpan.FromSeconds(60),
                    TimeSpan.FromSeconds(120),
                    TimeSpan.FromMinutes(5)
                })
                .ExecuteAsync(() => ApiCall());
            return result.StatusCode.ToString();
        }

        public async Task<RestResponse> ApiCall()
        {
            var options =
                new RestClientOptions(host)
                {
                    MaxTimeout = -1
                };
            var client = new RestClient(options);
            var request =
                new RestRequest(route,
                    Method.Post);

            var authString = Convert.ToBase64String(Encoding.UTF8.GetBytes(userId + ":" + password));
            request.AddHeader("Authorization",
                "Basic " + authString);
            request.AlwaysMultipartFormData = true;
            request.AddFile(fileType, filePath);
            var response = await client.ExecuteAsync(request);
            var responseContent = response.Content;
            return response;
        }
    }
}
2

The code inside the Execute method will instead be used in a message handler.

3

@JackInTheBoxBen NServiceBus will indeed retry the message for you. See the docs for more details. However, it won’t make any distinction between the status codes returned by the external API. To do that, you’d have to implement a custom recoverability policy. This is what I recommend, since that integrates with all the available recoverability mechanisms such as delayed retries, and for exceptions which are not worth retrying, you can immediately move the message to the error queue. If you use Polly, then any exceptions which are not worth retrying will then be handed over to the NServiceBus retry mechanism, which will retry redundantly. You could prevent that with a custom recoverability policy, but then if you are writing one of those then you may as well rely on NServiceBus recoverability in the first place.

With a custom recoverability policy, you can also use delayed retries, which will move the message back to the queue in the meantime, freeing up resources. If you use Polly, then retries will be bound to within a single call to a handler, and the any resources involved in that will be retained while your Polly retries are occurring.

1 Like
4

No problem at all to mix polly with NServiceBus its just that each retry/recovery method is slightly different. Polly can be super lightweight and is especially suitable for retrying a simple file/network operation without requiring to fully re-run the whole message processing pipeline.

That said, in most environments, this is likely a premature optimization. It also makes the code just a little bit more complex and harder to maintain. I would only add it if actually has a strong benefit on top of NServiceBus recoverability.

I would advise against using any delays in your Polly policies as while the retry policy is executed and waiting the concurrency slot is taken and not be available for any other messages in the queue.

2 Likes