Hi,
using a Asp.Net Web Forms application we have still the bug described here
This is the scenario:
we are including:
NServiceBus.Unity 6.2.1
NServiceBus 5.2.25
Unity 4.0.1
we are setting up the bus configuration in the Global.Application_Start event of the application, saying to use our existing UnityContainer (where we registered all our instances and types)
sometime the bus is disposed (I think by itself), for example when the RabbitMQ service crashes or when the Database configured as the Nservicebus Persistence is unreachable
when it happens, our UnityContainer will be disposed too and when our Application tries to access it, for resolving its instances and types, it will crash
I’ve been prepared a demo to explain the problem; here you can download it (I can’t upload it here because I’m a new user)
This is a demo:
it is a simple Asp.Net Web Forms application that configures a MSMQ transport and a inMemory persistence
if you click on the button “Execute action”, the application will use a registered type from the container to perform an action
if you click on the button “Simulate Bus Down”, the application will simulate the Bus dispose getting its instance and calling Bus.Dispose()
now, if you click again on the button “Execute action”, the application will raise an ObjectDisposedException on the object UnityObjectBuilder
There seem to be two problems here: The first being that your IBus instance is being disposed. The instance of IBus will never dispose itself, even if RabbitMQ is unavailable for whatever reason. If the transport is not available, NServiceBus should enter a circuit breaker state if underlying infrastructure is not available. I think it’s important to identify what is disposing of the IBus instance in your application.
The second problem is related to Unity somehow needing access to a disposed instance in order to construct a separate instance. I’m busy looking in to this for you and will let you know once I have an answer.
Hi William, thanks for you help, it is much appreciated.
Yes we have two problems and they are related because problem 1 raises problem 2.
Problem 1
It’s hard to define “when and because” Bus was disposed. I can assure you that we don’t dispose it in no situation.
These are the common situations that I’ve experienced in different projects; I know that they miss a lot of details but it’s hard to extabilish what is happing during these “excalation of problems”.
RabbitMQ 3.2.4 Windows Service was up and running and the Audit queue contained a lot of messages: looking at the log of my endpoints I saw a lot of connection error to RabbitMQ (see the following image) and the UnityContainer of my Web Backend (which contained an instance of Bus) was disposed.
For one Customer we have the persistence configured with NServicebus.NHibernate on a Oracle Database: when the Oracle Database is not reachable (it is a common situation when the Customer does maintenace on its database), the UnityContainer of my Web Backend (which contained an instance of Bus) is disposed.
2017-11-06 13:52:37.848 ERROR NServiceBus Repeated failures when fetching timeouts from storage, endpoint will be terminated. Oracle.ManagedDataAccess.Client.OracleException (0x80004005): ORA-12541: TNS: No listener —> OracleInternal.Network.NetworkException (0x80004005): ORA-12541: TNS: No listener —> System.Net.Sockets.SocketException (0x80004005): No connection could be made because the target machine actively refused it 155.66.167.42:1521
at System.Net.Sockets.Socket.EndConnect(IAsyncResult asyncResult)
at System.Net.Sockets.TcpClient.EndConnect(IAsyncResult asyncResult)
at OracleInternal.Network.TcpTransportAdapter.Connect(ConnectionOption conOption)
at OracleInternal.Network.OracleCommunication.DoConnect(String tnsDescriptor)
at OracleInternal.ServiceObjects.OracleConnectionImpl.Connect(ConnectionString cs, Boolean bOpenEndUserSession, CriteriaCtx criteriaCtx, String instanceName)
at OracleInternal.ConnectionPool.PoolManager3.Get(ConnectionString csWithDiffOrNewPwd, Boolean bGetForApp, CriteriaCtx criteriaCtx, String affinityInstanceName, Boolean bForceMatch) at OracleInternal.ConnectionPool.OraclePoolManager.Get(ConnectionString csWithNewPassword, Boolean bGetForApp, CriteriaCtx criteriaCtx, String affinityInstanceName, Boolean bForceMatch) at OracleInternal.ConnectionPool.OracleConnectionDispenser3.Get(ConnectionString cs, PM conPM, ConnectionString pmCS, SecureString securedPassword, SecureString securedProxyPassword, CriteriaCtx criteriaCtx)
at Oracle.ManagedDataAccess.Client.OracleConnection.Open()
at NHibernate.Connection.DriverConnectionProvider.GetConnection()
at NServiceBus.TimeoutPersisters.NHibernate.TimeoutPersister.GetNextChunk(DateTime startSlice, DateTime& nextTimeToRunQuery) in C:\BuildAgent\work\5135de308b2f3016\src\NServiceBus.NHibernate\TimeoutPersisters\TimeoutPersister.cs:line 43
at NServiceBus.Timeout.Hosting.Windows.TimeoutPersisterReceiver.Poll(Object obj) in C:\BuildAgent\work\3206e2123f54fce4\src\NServiceBus.Core\Timeout\Hosting\Windows\TimeoutPersisterReceiver.cs:line 90
at System.Threading.Tasks.Task.Execute()
For another Customer we have the persistence configured on RavenDB 3.5, and when it had an outage (at the moment i don’t have a log that shows the RavenDB problem), I saw the following error in a endpoint log and the UnityContainer of my Web Backend (which contained an instance of Bus) was disposed.
Application: NServiceBus.Host.exe
Framework Version: v4.0.30319
Description: The application requested process termination through System.Environment.FailFast(string message).
Message: The following critical error was encountered by NServiceBus:
Repeated failures when fetching timeouts from storage, endpoint will be terminated. NServiceBus is shutting down.
Stack:
at System.Environment.FailFast(System.String, System.Exception)
at System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
at System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
at System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem()
at System.Threading.ThreadPoolWorkQueue.Dispatch()
Problem 2
Ok, thanks. I’m waiting for your solution about Unity container dispose issue.
Still working on Problem 2. Current status is that it doesn’t actually seem to dispose the container, but the container is trying to load something it shouldn’t be.
Problem 1:
It seems that in V5 the bus is disposed in critical errors
