Application Fails while trying to load

ssingh · March 13, 2018, 3:25pm

I am trying to use RabbitMq as transport which is installed on a remote server,
so when I run Nservice bus application on same machine which has RabbitMq server then it works fine but fails while trying to connect remotely.

I have similar setup working in another application
user/password which I am specifying in “Nservicebus/Transport” key has administrator permission

I get following exception:
Exception thrown: ‘RabbitMQ.Client.Exceptions.BrokerUnreachableException’ in mscorlib.dll
Exception thrown: ‘RabbitMQ.Client.Exceptions.BrokerUnreachableException’ in YCM.DWH.EOD.Manager.Calculation.exe
System.Transactions Critical: 0 : Microsoft Learn: Build skills that open doors in your careerUnhandled exceptionYCM.DWH.EOD.Manager.Calculation.exeRabbitMQ.Client.Exceptions.BrokerUnreachableException, RabbitMQ.Client, Version=5.0.0.0, Culture=neutral, PublicKeyToken=89e7d7c5feba84ceNone of the specified endpoints were reachable at RabbitMQ.Client.ConnectionFactory.CreateConnection(IEndpointResolver endpointResolver, String clientProvidedName)
at RabbitMQ.Client.ConnectionFactory.CreateConnection(String clientProvidedName)
at NServiceBus.Transport.RabbitMQ.ConnectionFactory.CreateConnection(String connectionName) in C:\BuildAgent\work\a9e6741f41af7061\src\NServiceBus.RabbitMQ\Connection\ConnectionFactory.cs:line 74
at NServiceBus.Transport.RabbitMQ.QueueCreator.CreateQueueIfNecessary(QueueBindings queueBindings, String identity) in C:\BuildAgent\work\a9e6741f41af7061\src\NServiceBus.RabbitMQ\Administration\QueueCreator.cs:line 21
at NServiceBus.QueuesCreator.Install(String identity) in C:\BuildAgent\work\a93f853f0c1b9532\src\NServiceBus.Core\Unicast\Queuing\QueuesCreator.cs:line 27
at NServiceBus.InitializableEndpoint.<RunInstallers>d__13.MoveNext() in C:\BuildAgent\work\a93f853f0c1b9532\src\NServiceBus.Core\InitializableEndpoint.cs:line 184
— End of stack trace from previous location where exception was thrown —
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at NServiceBus.InitializableEndpoint.<Initialize>d__1.MoveNext() in C:\BuildAgent\work\a93f853f0c1b9532\src\NServiceBus.Core\InitializableEndpoint.cs:line 68
— End of stack trace from previous location where exception was thrown —
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at NServiceBus.Endpoint.<Start>d__1.MoveNext() in C:\BuildAgent\work\a93f853f0c1b9532\src\NServiceBus.Core\Endpoint.cs:line 27
— End of stack trace from previous location where exception was thrown —
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.TaskAwaiter1.GetResult() at YCM.DWH.EOD.Manager.Calculation.Program.&lt;Main&gt;d__1.MoveNext() in D:\WorkSpace\YCM\YCM.Core\Main\Departments\Operations\YCM.DWH.EOD\YCM.DWH.EOD.Manager.Calculation\Program.cs:line 64 --- End of stack trace from previous location where exception was thrown --- at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Runtime.CompilerServices.TaskAwaiter.GetResult() at YCM.DWH.EOD.Manager.Calculation.Program.&lt;Main&gt;(String[] args)</StackTrace><ExceptionString>RabbitMQ.Client.Exceptions.BrokerUnreachableException: None of the specified endpoints were reachable ---&gt; RabbitMQ.Client.Exceptions.ConnectFailureException: Connection failed ---&gt; System.Net.Sockets.SocketException: An attempt was made to access a socket in a way forbidden by its access permissions 192.168.109.19:5672 at System.Net.Sockets.Socket.DoConnect(EndPoint endPointSnapshot, SocketAddress socketAddress) at System.Net.Sockets.Socket.Connect(EndPoint remoteEP) at System.Net.Sockets.Socket.Connect(IPAddress address, Int32 port) at RabbitMQ.Client.TcpClientAdapter.&lt;ConnectAsync&gt;d__2.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at RabbitMQ.Client.Impl.TaskExtensions.&lt;TimeoutAfter&gt;d__1.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at RabbitMQ.Client.Impl.SocketFrameHandler.ConnectOrFail(ITcpClient socket, AmqpTcpEndpoint endpoint, Int32 timeout) --- End of inner exception stack trace --- at RabbitMQ.Client.EndpointResolverExtensions.SelectOne[T](IEndpointResolver resolver, Func2 selector)
at RabbitMQ.Client.Framing.Impl.AutorecoveringConnection.Init(IEndpointResolver endpoints)
at RabbitMQ.Client.ConnectionFactory.CreateConnection(IEndpointResolver endpointResolver, String clientProvidedName)
— End of inner exception stack trace —
at RabbitMQ.Client.ConnectionFactory.CreateConnection(IEndpointResolver endpointResolver, String clientProvidedName)
at RabbitMQ.Client.ConnectionFactory.CreateConnection(String clientProvidedName)
at NServiceBus.Transport.RabbitMQ.ConnectionFactory.CreateConnection(String connectionName) in C:\BuildAgent\work\a9e6741f41af7061\src\NServiceBus.RabbitMQ\Connection\ConnectionFactory.cs:line 74
at NServiceBus.Transport.RabbitMQ.QueueCreator.CreateQueueIfNecessary(QueueBindings queueBindings, String identity) in C:\BuildAgent\work\a9e6741f41af7061\src\NServiceBus.RabbitMQ\Administration\QueueCreator.cs:line 21
at NServiceBus.QueuesCreator.Install(String identity) in C:\BuildAgent\work\a93f853f0c1b9532\src\NServiceBus.Core\Unicast\Queuing\QueuesCreator.cs:line 27
at NServiceBus.InitializableEndpoint.<RunInstallers>d__13.MoveNext() in C:\BuildAgent\work\a93f853f0c1b9532\src\NServiceBus.Core\InitializableEndpoint.cs:line 184
— End of stack trace from previous location where exception was thrown —
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at NServiceBus.InitializableEndpoint.<Initialize>d__1.MoveNext() in C:\BuildAgent\work\a93f853f0c1b9532\src\NServiceBus.Core\InitializableEndpoint.cs:line 68
— End of stack trace from previous location where exception was thrown —
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at NServiceBus.Endpoint.<Start>d__1.MoveNext() in C:\BuildAgent\work\a93f853f0c1b9532\src\NServiceBus.Core\Endpoint.cs:line 27
— End of stack trace from previous location where exception was thrown —
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.TaskAwaiter1.GetResult() at YCM.DWH.EOD.Manager.Calculation.Program.&lt;Main&gt;d__1.MoveNext() in D:\WorkSpace\YCM\YCM.Core\Main\Departments\Operations\YCM.DWH.EOD\YCM.DWH.EOD.Manager.Calculation\Program.cs:line 64 --- End of stack trace from previous location where exception was thrown --- at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Runtime.CompilerServices.TaskAwaiter.GetResult() at YCM.DWH.EOD.Manager.Calculation.Program.&lt;Main&gt;(String[] args)</ExceptionString><InnerException><ExceptionType>RabbitMQ.Client.Exceptions.ConnectFailureException, RabbitMQ.Client, Version=5.0.0.0, Culture=neutral, PublicKeyToken=89e7d7c5feba84ce</ExceptionType><Message>Connection failed</Message><StackTrace> at RabbitMQ.Client.EndpointResolverExtensions.SelectOne[T](IEndpointResolver resolver, Func2 selector)
at RabbitMQ.Client.Framing.Impl.AutorecoveringConnection.Init(IEndpointResolver endpoints)
at RabbitMQ.Client.ConnectionFactory.CreateConnection(IEndpointResolver endpointResolver, String clientProvidedName)RabbitMQ.Client.Exceptions.ConnectFailureException: Connection failed —> System.Net.Sockets.SocketException: An attempt was made to access a socket in a way forbidden by its access permissions 192.168.109.19:5672
at System.Net.Sockets.Socket.DoConnect(EndPoint endPointSnapshot, SocketAddress socketAddress)
at System.Net.Sockets.Socket.Connect(EndPoint remoteEP)
at System.Net.Sockets.Socket.Connect(IPAddress address, Int32 port)
at RabbitMQ.Client.TcpClientAdapter.<ConnectAsync>d__2.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at RabbitMQ.Client.Impl.TaskExtensions.<TimeoutAfter>d__1.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at RabbitMQ.Client.Impl.SocketFrameHandler.ConnectOrFail(ITcpClient socket, AmqpTcpEndpoint endpoint, Int32 timeout)
— End of inner exception stack trace —
at RabbitMQ.Client.EndpointResolverExtensions.SelectOne[T](IEndpointResolver resolver, Func`2 selector)
at RabbitMQ.Client.Framing.Impl.AutorecoveringConnection.Init(IEndpointResolver endpoints)
at RabbitMQ.Client.ConnectionFactory.CreateConnection(IEndpointResolver endpointResolver, String clientProvidedName)System.Net.Sockets.SocketException, System, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089An attempt was made to access a socket in a way forbidden by its access permissions 192.168.109.19:5672 at System.Net.Sockets.Socket.DoConnect(EndPoint endPointSnapshot, SocketAddress socketAddress)
at System.Net.Sockets.Socket.Connect(EndPoint remoteEP)
at System.Net.Sockets.Socket.Connect(IPAddress address, Int32 port)
at RabbitMQ.Client.TcpClientAdapter.<ConnectAsync>d__2.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at RabbitMQ.Client.Impl.TaskExtensions.<TimeoutAfter>d__1.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at RabbitMQ.Client.Impl.SocketFrameHandler.ConnectOrFail(ITcpClient socket, AmqpTcpEndpoint endpoint, Int32 timeout)System.Net.Sockets.SocketException (0x80004005): An attempt was made to access a socket in a way forbidden by its access permissions 192.168.109.19:5672
at System.Net.Sockets.Socket.DoConnect(EndPoint endPointSnapshot, SocketAddress socketAddress)
at System.Net.Sockets.Socket.Connect(EndPoint remoteEP)
at System.Net.Sockets.Socket.Connect(IPAddress address, Int32 port)
at RabbitMQ.Client.TcpClientAdapter.<ConnectAsync>d__2.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at RabbitMQ.Client.Impl.TaskExtensions.<TimeoutAfter>d__1.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at RabbitMQ.Client.Impl.SocketFrameHandler.ConnectOrFail(ITcpClient socket, AmqpTcpEndpoint endpoint, Int32 timeout)271D
An unhandled exception of type ‘RabbitMQ.Client.Exceptions.BrokerUnreachableException’ occurred in mscorlib.dll
None of the specified endpoints were reachable

andreasohlund · March 14, 2018, 7:36am

My hunch is that you need to open up your firewall, can you confirm that port 5672 is open?

See Networking and RabbitMQ — RabbitMQ for more details

ssingh · March 14, 2018, 1:19pm

Yes, because when I try to do same thing from another application then it works fine, It fails for few specific applications.

andreasohlund · March 14, 2018, 1:32pm

Are all those apps on the same server? do the all run using the same user account?

ssingh · March 19, 2018, 7:27pm

Sorry for not reply, here is the error that I get now
2018-03-19 15:26:28.740 ERROR YCM.DWH.EOD.Manager.ExceptionReport.Program RabbitMQ.Client.Exceptions.BrokerUnreachableException: None of the specified endpoints were reachable —> RabbitMQ.Client.Exceptions.ConnectFailureException: Connection failed —> System.Net.Sockets.SocketException: An attempt was made to access a socket in a way forbidden by its access permissions 192.168.109.19:5672
at System.Net.Sockets.Socket.DoConnect(EndPoint endPointSnapshot, SocketAddress socketAddress)
at System.Net.Sockets.Socket.Connect(EndPoint remoteEP)
at System.Net.Sockets.Socket.Connect(IPAddress address, Int32 port)
at RabbitMQ.Client.TcpClientAdapter.d__2.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at RabbitMQ.Client.Impl.TaskExtensions.d__1.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at RabbitMQ.Client.Impl.SocketFrameHandler.ConnectOrFail(ITcpClient socket, AmqpTcpEndpoint endpoint, Int32 timeout)
— End of inner exception stack trace —
at RabbitMQ.Client.EndpointResolverExtensions.SelectOne[T](IEndpointResolver resolver, Func2 selector) at RabbitMQ.Client.Framing.Impl.AutorecoveringConnection.Init(IEndpointResolver endpoints) at RabbitMQ.Client.ConnectionFactory.CreateConnection(IEndpointResolver endpointResolver, String clientProvidedName) --- End of inner exception stack trace --- at RabbitMQ.Client.ConnectionFactory.CreateConnection(IEndpointResolver endpointResolver, String clientProvidedName) at RabbitMQ.Client.ConnectionFactory.CreateConnection(String clientProvidedName) at NServiceBus.Transport.RabbitMQ.ConnectionFactory.CreateConnection(String connectionName) in C:\BuildAgent\work\a9e6741f41af7061\src\NServiceBus.RabbitMQ\Connection\ConnectionFactory.cs:line 74 at NServiceBus.Transport.RabbitMQ.QueueCreator.CreateQueueIfNecessary(QueueBindings queueBindings, String identity) in C:\BuildAgent\work\a9e6741f41af7061\src\NServiceBus.RabbitMQ\Administration\QueueCreator.cs:line 21 at NServiceBus.QueuesCreator.Install(String identity) in C:\BuildAgent\work\a93f853f0c1b9532\src\NServiceBus.Core\Unicast\Queuing\QueuesCreator.cs:line 27 at NServiceBus.InitializableEndpoint.<RunInstallers>d__13.MoveNext() in C:\BuildAgent\work\a93f853f0c1b9532\src\NServiceBus.Core\InitializableEndpoint.cs:line 184 --- End of stack trace from previous location where exception was thrown --- at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at NServiceBus.InitializableEndpoint.<Initialize>d__1.MoveNext() in C:\BuildAgent\work\a93f853f0c1b9532\src\NServiceBus.Core\InitializableEndpoint.cs:line 68 --- End of stack trace from previous location where exception was thrown --- at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at NServiceBus.Endpoint.<Start>d__1.MoveNext() in C:\BuildAgent\work\a93f853f0c1b9532\src\NServiceBus.Core\Endpoint.cs:line 27 --- End of stack trace from previous location where exception was thrown --- at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Runtime.CompilerServices.TaskAwaiter1.GetResult()
at YCM.DWH.EOD.Manager.ExceptionReport.Program.d__1.MoveNext() in D:\WorkSpace\YCM\YCM.Core\Main\Departments\Operations\YCM.DWH.EOD\YCM.DWH.EOD.Manager.ExceptionReport\Program.cs:line 57

andreasohlund · March 20, 2018, 7:12am

That is a sure sign that there is some firewall interfering. Would you be able to verify that TCP communication over port 5672 is allowed btw the involved machines?

ssingh · March 20, 2018, 12:20pm

Yes, because it works from another application with same settings but different Queue name

ssingh · March 20, 2018, 2:51pm

You guys were right, it turned out tobe Firewall issue… Thanks