Using existing unitycontainer and getting object disposed error for UnityObjectBuilder

unity

(Carlo Fedeli) #1

Hi,
using a Asp.Net Web Forms application we have still the bug described here

This is the scenario:

  • we are including:
    • NServiceBus.Unity 6.2.1
    • NServiceBus 5.2.25
    • Unity 4.0.1
  • we are setting up the bus configuration in the Global.Application_Start event of the application, saying to use our existing UnityContainer (where we registered all our instances and types)
  • sometime the bus is disposed (I think by itself), for example when the RabbitMQ service crashes or when the Database configured as the Nservicebus Persistence is unreachable
  • when it happens, our UnityContainer will be disposed too and when our Application tries to access it, for resolving its instances and types, it will crash

I’ve been prepared a demo to explain the problem; here you can download it (I can’t upload it here because I’m a new user)
This is a demo:

  • it is a simple Asp.Net Web Forms application that configures a MSMQ transport and a inMemory persistence
  • if you click on the button “Execute action”, the application will use a registered type from the container to perform an action
  • if you click on the button “Simulate Bus Down”, the application will simulate the Bus dispose getting its instance and calling Bus.Dispose()
  • now, if you click again on the button “Execute action”, the application will raise an ObjectDisposedException on the object UnityObjectBuilder

Can you help me please?

C


(William Brander) #2

Hi Carlo

There seem to be two problems here: The first being that your IBus instance is being disposed. The instance of IBus will never dispose itself, even if RabbitMQ is unavailable for whatever reason. If the transport is not available, NServiceBus should enter a circuit breaker state if underlying infrastructure is not available. I think it’s important to identify what is disposing of the IBus instance in your application.

The second problem is related to Unity somehow needing access to a disposed instance in order to construct a separate instance. I’m busy looking in to this for you and will let you know once I have an answer.


(Carlo Fedeli) #3

Hi William, thanks for you help, it is much appreciated.
Yes we have two problems and they are related because problem 1 raises problem 2.

Problem 1
It’s hard to define “when and because” Bus was disposed. I can assure you that we don’t dispose it in no situation.
These are the common situations that I’ve experienced in different projects; I know that they miss a lot of details but it’s hard to extabilish what is happing during these “excalation of problems”.

  • RabbitMQ 3.2.4 Windows Service was up and running and the Audit queue contained a lot of messages: looking at the log of my endpoints I saw a lot of connection error to RabbitMQ (see the following image) and the UnityContainer of my Web Backend (which contained an instance of Bus) was disposed.

  • For one Customer we have the persistence configured with NServicebus.NHibernate on a Oracle Database: when the Oracle Database is not reachable (it is a common situation when the Customer does maintenace on its database), the UnityContainer of my Web Backend (which contained an instance of Bus) is disposed.

    2017-11-06 13:52:37.848 ERROR NServiceBus Repeated failures when fetching timeouts from storage, endpoint will be terminated.
    Oracle.ManagedDataAccess.Client.OracleException (0x80004005): ORA-12541: TNS: No listener —> OracleInternal.Network.NetworkException (0x80004005): ORA-12541: TNS: No listener —> System.Net.Sockets.SocketException (0x80004005): No connection could be made because the target machine actively refused it 155.66.167.42:1521
    at System.Net.Sockets.Socket.EndConnect(IAsyncResult asyncResult)
    at System.Net.Sockets.TcpClient.EndConnect(IAsyncResult asyncResult)
    at OracleInternal.Network.TcpTransportAdapter.Connect(ConnectionOption conOption)
    at OracleInternal.Network.OracleCommunication.DoConnect(String tnsDescriptor)
    at OracleInternal.ServiceObjects.OracleConnectionImpl.Connect(ConnectionString cs, Boolean bOpenEndUserSession, CriteriaCtx criteriaCtx, String instanceName)
    at OracleInternal.ConnectionPool.PoolManager3.Get(ConnectionString csWithDiffOrNewPwd, Boolean bGetForApp, CriteriaCtx criteriaCtx, String affinityInstanceName, Boolean bForceMatch) at OracleInternal.ConnectionPool.OraclePoolManager.Get(ConnectionString csWithNewPassword, Boolean bGetForApp, CriteriaCtx criteriaCtx, String affinityInstanceName, Boolean bForceMatch) at OracleInternal.ConnectionPool.OracleConnectionDispenser3.Get(ConnectionString cs, PM conPM, ConnectionString pmCS, SecureString securedPassword, SecureString securedProxyPassword, CriteriaCtx criteriaCtx)
    at Oracle.ManagedDataAccess.Client.OracleConnection.Open()
    at NHibernate.Connection.DriverConnectionProvider.GetConnection()
    at NServiceBus.TimeoutPersisters.NHibernate.TimeoutPersister.GetNextChunk(DateTime startSlice, DateTime& nextTimeToRunQuery) in C:\BuildAgent\work\5135de308b2f3016\src\NServiceBus.NHibernate\TimeoutPersisters\TimeoutPersister.cs:line 43
    at NServiceBus.Timeout.Hosting.Windows.TimeoutPersisterReceiver.Poll(Object obj) in C:\BuildAgent\work\3206e2123f54fce4\src\NServiceBus.Core\Timeout\Hosting\Windows\TimeoutPersisterReceiver.cs:line 90
    at System.Threading.Tasks.Task.Execute()

  • For another Customer we have the persistence configured on RavenDB 3.5, and when it had an outage (at the moment i don’t have a log that shows the RavenDB problem), I saw the following error in a endpoint log and the UnityContainer of my Web Backend (which contained an instance of Bus) was disposed.

    Application: NServiceBus.Host.exe
    Framework Version: v4.0.30319
    Description: The application requested process termination through System.Environment.FailFast(string message).
    Message: The following critical error was encountered by NServiceBus:
    Repeated failures when fetching timeouts from storage, endpoint will be terminated.
    NServiceBus is shutting down.
    Stack:
    at System.Environment.FailFast(System.String, System.Exception)
    at System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
    at System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
    at System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem()
    at System.Threading.ThreadPoolWorkQueue.Dispatch()

Problem 2
Ok, thanks. I’m waiting for your solution about Unity container dispose issue.


(William Brander) #4

Still working on Problem 2. Current status is that it doesn’t actually seem to dispose the container, but the container is trying to load something it shouldn’t be.

Problem 1:

It seems that in V5 the bus is disposed in critical errors

So your second customer has implemented the recommended fix for if the environment fails: https://docs.particular.net/nservicebus/hosting/critical-errors?version=core_5#a-possible-custom-implementation

It probably makes sense to implement something similar in your current example.


(William Brander) #5

I found the reason for Problem 1.

I’ve created an issue here. I’ll ping you here too once a fix is ready.


(William Brander) #6

NServiceBus.Unity 6.2.2 has been released and should fix Problem 1.

Can you give it a try please and let me know if it works?


(Carlo Fedeli) #7

Do you mean Problem 2 - Dispose Unity Container when Bus is disposed?


(William Brander) #8

Ah yes, I seem to have confused myself :slight_smile: