Unity multi-tenant configuration

1

Hi

I’m new to NServiceBus and am currently working on a POC to migrate one of our sync services to test things out. Our system is multi-tenant and my idea was to have a base type that every command passes containing the tenant context.

	public class MessageBase
{
	public TenantContext TenantContext { get; set; }
}

Our cloud solution exists in what we call “pods”, and a pod will have a database containing all the tenant details, settings and connection strings to their individual databases. So we retrieve the tenant details when the sync item is first added to the queue to be processed and then this TenantContext can be used to resolve the TenantContextFactory which most services, including the ITeanantDbConnection will rely on.

My current POC contains 7 endpoints, which would all need to resolve the unity container with a TenantContext.

As we retrieve the TenantContext when we handle a message, how would I be best to go about resolving the TenantContext for the TenantContextFactory?

We do something similar for the API, but that is using the HttpContext:

		public static void Register(IUnityContainer container)
	{
		container.RegisterType<TenantContextFactory>(new ContainerControlledLifetimeManager(), new InjectionFactory(u =>
		{
			return new TenantContextFactory(() => u.Resolve<TenantContext>());
		}));
		container.RegisterType<TenantContext>(new InjectionFactory(GetContext));
	}

	private static TenantContext GetContext(IUnityContainer container)
	{
		if (HttpContext.Current != null)
		{
			var tenantGuid = HttpContext.Current.Request.Headers["X-Tenant"];
			return string.IsNullOrEmpty(tenantGuid) ? 
				null :
				GetContextFromCache(tenantGuid, container);
		}

		return null;
	}

	private static TenantContext GetContextFromCache(string guid, IUnityContainer container)
	{
		var cache = MemoryCache.Default;
		var key = $"tenantcontext_{guid}";
		var cachedContext = cache.Get(key) as TenantContext;
		if (cachedContext != null)
			return cachedContext;

		var repo = container.Resolve<ITenantAdminRepository>();
		var tenant = repo.GetTenant(Guid.Parse(guid));

		if (tenant == null)
			return null;

		var context = new TenantContext
		{
			ConnectionString = tenant.ConnectionString,
			TenantGuid = tenant.TenantGuid,
			TenantName = tenant.Name,
			Logging_LogErrors = true,
			Logging_LogInfo = false,
			Logging_LogDebug = false
		};

		var policy = new CacheItemPolicy
		{
			AbsoluteExpiration = DateTime.UtcNow.AddHours(10),
			Priority = CacheItemPriority.Default
		};

		cache.Add(key, context, policy);

		return context;
	}

I wonder if there is some way we can hook in to the endpoint pipeline and retrieve either the TenantContext for the command and resolve the dependencies ahead of the request ending up in the handler. Open to better ways around this I am completely new to NSeviceBus so there may be some functionality that helps with this?

Thanks in advance

Andy

2

I wonder if there is some way we can hook in to the endpoint pipeline and retrieve either the TenantContext for the command and resolve the dependencies ahead of the request ending up in the handler

NServiceBus allows you to register custom “behaviors” into the pipeline which will be invoked when processing an incoming message. Have a look at our documentation pages about the pipeline, especially this page might be interesting for you: Manipulate pipeline with behaviors • NServiceBus • Particular Docs

So what you roughly could do is something like this:

  • In a behavior, retrieve the TenantContext from the message properties. Btw. since the tenant information aren’t related with the business data, you might want to consider adding these information to the message headers instead.
  • Inject the “TenantContextFactory” into the behavior which you will have to bind in a “unit of work scope” (see more about this here: Dependency Injection • NServiceBus • Particular Docs) to ensure that every message will get it’s own factory.
  • Set the retrieved tenant context on the factory. Obviously this all needs to happen before you try to resolve something from the factory the first time, but the behaviors are guaranteed to run before your message handlers, so that shouldn’t be an issue in most cases.
  • You can then inject the TenantContextFactory in your handlers and invoke it, as it has been initialized with the messages information by the behavior.

There are other approaches (where the simplest one would probably be to just read the information from the message when invoking the factory and passing it as a parameter), but this is a very basic and generic approach which works with all DI containers.

btw. I found the “TenantContextFactory” naming a bit confusing, as this would usually indicate this is a factory creating TenantContext instances, which doesn’t seem to be the case if I understand your explanation correctly.

Warning: this approach only works for your custom database connections. If you need to use different databases for sagas, timeouts or other “built-in” storages, this will require more work.

There is also a sample about multi-tenant endpoints (also addressing the warning I mentioned above), but for NHibernate persistence, but this might still be interesting for you to look at: https://docs.particular.net/samples/outbox/multi-tenant/?version=core_7

3

Hi Tim,

Sorry for taking so long to get back and thanks for the detailed, response!

So I’ve got things working a little more since I first posted and I’m not taking in to account some of your suggestions but have a few question about how best to implement them…

So just so you understand how I’ve got things working at them moment, I pass the TenantContext around currently just as a message base, but I would like to move move this to the message header (I’ll come back to this).

So I’ve updated my unity config as follows:

		private static void RegisterFactories(IUnityContainer container)
		{
			container.RegisterType<TenantContextFactory>(new ContainerControlledLifetimeManager(), new InjectionFactory(u =>
			{
				return new TenantContextFactory(() => u.Resolve<TenantContext>());
			}));
			container.RegisterType<PersonContextFactory>(new ContainerControlledLifetimeManager(), new InjectionFactory(u =>
			{
				return new PersonContextFactory(() => u.Resolve<PersonSecurityContext>());
			}));
		}

		private static void RegisterContext(IUnityContainer container)
		{
			container.RegisterType<TenantContext>();
			container.RegisterType<TenantSettings>(new InjectionFactory(GetSettings));
			container.RegisterType<PersonSecurityContext>(new InjectionFactory(u => new PersonSecurityContext()));
			container.RegisterType<IApplicationAccessor>(new InjectionFactory(GetApplicationAccessorFromCache));
			container.RegisterType<ISessionAccessor>(new InjectionFactory(u => null));
		}

Then in my handlers I inject a TenantContextFactory (I know this naming doesn’t make sense, sadly its some legacy code in our solution and is used in most services). I then take my TenantContext from the message and have extended the TenantContextFactory to be mutable.

	public class TenantContextFactory
{
	private Func<TenantContext> _contextAction;

	public TenantContextFactory(Func<TenantContext> container)
	{
		_contextAction = container;
	}

	public void CreateTenantContext(Func<TenantContext> container)
	{
		_contextAction = container;
	}

	public TenantContext GetTenantContext()
	{
		return _contextAction();
	}
}

So in the handler (*I want to move this to a behaviour), I then pass in the the TenantContext and this allows me to resolve the tenant at runtime.

_tenantContextFactory.CreateTenantContext(() => tenantContext);

This seems to work, but I feel there should be a better way especially as the tenant context “factory” is now mutable.

I’ve started trying to move this logic to a behaviour and I’ve put the tenant context on a header now. Not sure if there is a better way, but I’ve serialised the TenantContext to a Json string to set it on the header, I still need to encrypt this as it contains a database connection.

var tenantRepo = container.Resolve<ITenantAdminRepository>();
			var tenant = tenantRepo.GetTenant(tenantGuid);
			var tenantContext = new TenantContext {ConnectionString = tenant.ConnectionString, TenantGuid = tenant.TenantGuid, TenantName = tenant.Name};

			// TODO: Add encryption to protect db string
			var sendOptions = new SendOptions();
			sendOptions.SetHeader("TenantContext", JsonConvert.SerializeObject(tenantContext));

As this gets passed in the header for every message I thought a message mutator could be used so I’ve got this:

public Task MutateOutgoing(MutateOutgoingMessageContext context)
	{
		if (context.TryGetIncomingMessage(out var incomingMessage))
		{
			context.OutgoingMessage = incomingMessage as IMessageBase;
		}

		if (context.TryGetIncomingHeaders(out var incomingHeaders))
		{
			context.OutgoingHeaders.Add("TenantContext", incomingHeaders["TenantContext"]);
		}

		return Task.CompletedTask;
	}

And then to handle the dependency resolution for the handlers I’ve got a behaviour as follows, but I’m not sure I’ve done this correctly:

	public class TenantContextBehavior : Behavior<IIncomingPhysicalMessageContext>
	{
		private readonly TenantSessionProvider _tenantSessionProvider;
		private readonly TenantContextFactory _tenantContextFactory;

		public TenantContextBehavior(TenantSessionProvider tenantSessionProvider, TenantContextFactory tenantContextFactory)
		{
			_tenantSessionProvider = tenantSessionProvider;
			_tenantContextFactory = tenantContextFactory;
		}

		// This pulls the tenant context JSON from the header, deserializes it and creates a tenant context factory that is accesible in the handler.
		public override async Task Invoke(IIncomingPhysicalMessageContext context, Func<Task> next)
		{
			var tenant = context.MessageHeaders["TenantContext"];
			var tenantContext = string.IsNullOrEmpty(tenant) ? new TenantContext() : JsonConvert.DeserializeObject<TenantContext>(tenant);
			_tenantContextFactory.CreateTenantContext(() => tenantContext);

			using (var session = await _tenantSessionProvider.Open(_tenantContextFactory).ConfigureAwait(false))
			{
				context.Extensions.Set<ITenantSession>(session);

				try
				{
					await next().ConfigureAwait(false);

					await session.Commit().ConfigureAwait(false);
				}
				catch (Exception)
				{
					await session.Rollback().ConfigureAwait(false);

					throw;
				}
			}

			await next().ConfigureAwait(false);
		}

I was a little unsure about the Uow implementation I’ve had a look at the samples but wasn’t sure the best way to wire this up?

I was looking at outboxing, for our on premise customers we don’t have control of their environments so enabling DTC and managing it would be problematic.

In terms of saga storage I am using the tenant database, not the individual tenant’s database at the moment, as this is where we currently queue all tasks for all tenants to be processed.

Many thanks

Andy

4

Hey Andrew,

I’m not exactly sure how to correctly register the TenantContenxtFactory in via the unity API directly but I’d instead configure it like this:

endpointConfiguration.RegisterComponents(c => c.ConfigureComponent<TenantContextFactory>(DependencyLifecycle.InstancePerUnitOfWork));

This ensures a new TenantContextFactory is used for every message. This API also has overloads to define a custom object factory. Seeing the implementation of TenantContextFactory you posted, I have a few questions regarding that class:

  • Why do you pass a function in the ctor? The factory will always require “initialization” by the behavior before being able to resolve anything.
  • Why are you using a Func<TenantContext> if you could substitute this directly with TenantContext?

Is this the real implementation and purpose of your TenantContextFactory? If so, it looks like it would be much easier to just use an AsyncLocal<TenantContext> which you can also set via behavior and then access from anywhere within the call hierarchy.

Not sure if there is a better way, but I’ve serialised the TenantContext to a Json string to set it on the header, I still need to encrypt this as it contains a database connection.

I wasn’t fully considering that TenantContext is a class containing many properties. In that case I’d probably retract my recommendation to use headers (although your approach works). If you plan to encrypt the values, a message property might make sense as you could use MessagePropertyEncryption: Message Property Encryption • Particular Docs

your behavior implementation looks correct to me at first glance (aside from the previously asked question regarding the function parameter to the factory). Using the context.Extensions bag to pass down data in the pipeline is the easiest and recommended approach.