Large number of Retries

We have an issue where sending messages (ICommand) via a simple Send command generates a lot of retries. The retries are caused by System.InvalidOperationException: No handlers could be found for message type: <…> at NServiceBus.LoadHandlersConnector.d__1.MoveNext() in C:\BuildAgent\work\ed946b9f0e4aae01\src\NServiceBus.Core\Pipeline\Incoming\LoadHandlersConnector.cs:line 35

Eventually all the messages get processed as long the number of delayed retries is high enough. These info/warning messages are the same as when I stop the handler service altogether, except, of course, if the service is stopped the messages would eventually fail to process and get moved into the error queue.

I am wondering if we are missing certain configuration or the service/endpoints are not wired correctly. The number of retries needed for the messages to get through the system, does seem to depend on how busy the system is and how long does one handler needs to process the message (most of it are database insert/update operations). We are using NServiceBus 4.7.2 and SqlServer as the Transport type.