Allow password rotation without endpoint restart

,
1

Hi everyone

we rotate our database passwords on a schedule and would like to load the updated secret on the fly without requiring an endpoint restart. I was wondering what the suggest approach was to achieve this.

I found one way to do it is by trying to establish a connection to the database as part of the connection builder delegate. If the connection fails, reload the secret and try again. This seems expensive and not very efficient and I was wondering if there was a better way.

persistence.ConnectionBuilder(
() =>
{
	var connectionString = connectionStringDelegate();
	var connection = new MySqlConnection(connectionString);
	if (connection.CanConnect())
	{
		return connection;
	}

	// If the connection failed reload the connection string from AWS Secret
	// and try again.
	configuration.ReloadConfigurationProvider<AwsSecretsManagerConfigurationProvider>();

	connectionString = connectionStringDelegate();
	connection = new MySqlConnection(connectionString);

	return connection;
});

Serge

2

Hi @sglaeserpc,

For most aspects of NServiceBus, for example the connection information for a transport, you would be out of luck, as that stuff has to be loaded once at endpoint startup.

But with SQL Persistence you get a bit of a break as, as you’ve shown, it has the ConnectionBuilder delegate that’s called basically for each processed message.

The better way to architect that is probably to use a distributed cache (like Redis) or a distributed configuration system (examples include etcd, Zookeeper, and Consul) that is capable of pushing out new configuration data. Then your delegate would always use the value from the distribution store and not realize that it had actually just changed.

Or, rather than rotating a password, some services will support managed identity so the authentication is based on the identity of the process, but I’m not sure if MySQL supports that.