Message retry from ServiceInsight failing, PFB message header MessageHeader & ServiceControlLog for error log from ServiceControl. Using SQL server transport. From ServiceControl, it trying to search Queuename with “dbo” schema but in our case all queuename is in “nsb” schema
NServiceBus : 5.2.14
NServiceBus.Host : 6.0.0
NServiceBus.SqlServer : 2.2.3
[Not allowing to upload files so pasted below]
--------------------Message header--------------------------------------------
{“NServiceBus.MessageId”:"69b815a8-bf4f-48da-beef-a93101048cff
“NServiceBus.CorrelationId”:"686a6f9c-d654-4136-a1cb-a93101048d54
“NServiceBus.MessageIntent”:"Publish
“NServiceBus.Version”:"5.2.14
“NServiceBus.TimeSent”:"2018-08-03 10:20:23:961602 Z
“NServiceBus.ContentType”:"application/json
“NServiceBus.EnclosedMessageTypes”:"ABC.Contracts.Events.IEventPublshed, ABC.Contracts, Version=2018.3.1.437, Culture=neutral, PublicKeyToken=null
“NServiceBus.RelatedTo”:"97a3ecf1-859f-4042-ae86-a931010480ff
“NServiceBus.ConversationId”:"a77480ab-ea8c-4050-bb08-a9310103df76
“WinIdName”:"HQINTL1\310827
“NServiceBus.SqlServer.CallbackQueue”:"ABC.Endpoints.PQR.XXXXXXXX
“NServiceBus.OriginatingMachine”:"XXXXXXXX
“NServiceBus.OriginatingEndpoint”:"ABC.Endpoints.PQR
“$.diagnostics.originating.hostid”:"322b862a6f48d5c5ebb3ff2231df08e6
“NServiceBus.ReplyToAddress”:"ABC.Endpoints.PQR
“$.diagnostics.hostid”:"b28d1cd1c6ac07318ed8a794ed495575
“$.diagnostics.hostdisplayname”:"XXXXXXXX
“NServiceBus.ExceptionInfo.ExceptionType”:"System.Exception
“NServiceBus.ExceptionInfo.HelpLink”:null,“NServiceBus.ExceptionInfo.Message”:"test
“NServiceBus.ExceptionInfo.Source”:"ABC.Managers.Manager
“NServiceBus.ExceptionInfo.StackTrace”:"System.Exception: test\r\n at ABC.Managers.AAASaga.Handle(IEventPublshed message) in C:\Workspace\ABC2\branch\src\Managers\ABC.Managers.Manager\AAASaga.cs:line 36\r\n at NServiceBus.Unicast.MessageHandlerRegistry.Invoke(Object handler, Object message, Dictionary2 dictionary) in C:\\BuildAgent\\work\\3206e2123f54fce4\\src\\NServiceBus.Core\\Unicast\\MessageHandlerRegistry.cs:line 124\r\n at NServiceBus.InvokeHandlersBehavior.Invoke(IncomingContext context, Action next) in C:\\BuildAgent\\work\\3206e2123f54fce4\\src\\NServiceBus.Core\\Unicast\\Behaviors\\InvokeHandlersBehavior.cs:line 23\r\n at NServiceBus.BehaviorChain
1.InvokeNext(T context) in C:\BuildAgent\work\3206e2123f54fce4\src\NServiceBus.Core\Pipeline\BehaviorChain.cs:line 107\r\n at NServiceBus.BehaviorChain1.<>c__DisplayClass4_0.<InvokeNext>b__0() in C:\\BuildAgent\\work\\3206e2123f54fce4\\src\\NServiceBus.Core\\Pipeline\\BehaviorChain.cs:line 95\r\n at NServiceBus.SagaPersistenceBehavior.Invoke(IncomingContext context, Action next) in C:\\BuildAgent\\work\\3206e2123f54fce4\\src\\NServiceBus.Core\\Sagas\\SagaPersistenceBehavior.cs:line 90\r\n at NServiceBus.BehaviorChain
1.InvokeNext(T context) in C:\BuildAgent\work\3206e2123f54fce4\src\NServiceBus.Core\Pipeline\BehaviorChain.cs:line 107\r\n at NServiceBus.BehaviorChain1.<>c__DisplayClass4_0.<InvokeNext>b__0() in C:\\BuildAgent\\work\\3206e2123f54fce4\\src\\NServiceBus.Core\\Pipeline\\BehaviorChain.cs:line 95\r\n at NServiceBus.SetCurrentMessageBeingHandledBehavior.Invoke(IncomingContext context, Action next) in C:\\BuildAgent\\work\\3206e2123f54fce4\\src\\NServiceBus.Core\\Unicast\\Behaviors\\SetCurrentMessageBeingHandledBehavior.cs:line 17\r\n at NServiceBus.BehaviorChain
1.InvokeNext(T context) in C:\BuildAgent\work\3206e2123f54fce4\src\NServiceBus.Core\Pipeline\BehaviorChain.cs:line 107\r\n at NServiceBus.BehaviorChain1.<>c__DisplayClass4_0.<InvokeNext>b__0() in C:\\BuildAgent\\work\\3206e2123f54fce4\\src\\NServiceBus.Core\\Pipeline\\BehaviorChain.cs:line 95\r\n at NServiceBus.LoadHandlersBehavior.Invoke(IncomingContext context, Action next) in C:\\BuildAgent\\work\\3206e2123f54fce4\\src\\NServiceBus.Core\\Unicast\\Behaviors\\LoadHandlersBehavior.cs:line 45\r\n at NServiceBus.BehaviorChain
1.InvokeNext(T context) in C:\BuildAgent\work\3206e2123f54fce4\src\NServiceBus.Core\Pipeline\BehaviorChain.cs:line 107\r\n at NServiceBus.BehaviorChain1.<>c__DisplayClass4_0.<InvokeNext>b__0() in C:\\BuildAgent\\work\\3206e2123f54fce4\\src\\NServiceBus.Core\\Pipeline\\BehaviorChain.cs:line 95\r\n at NServiceBus.Transports.SQLServer.ReadIncomingCallbackAddressBehavior.Invoke(IncomingContext context, Action next) in C:\\BuildAgent\\work\\888dd942b44bad5a\\src\\NServiceBus.SqlServer\\ReadIncomingCallbackAddressBehavior.cs:line 18\r\n at NServiceBus.BehaviorChain
1.InvokeNext(T context) in C:\BuildAgent\work\3206e2123f54fce4\src\NServiceBus.Core\Pipeline\BehaviorChain.cs:line 107\r\n at NServiceBus.BehaviorChain1.<>c__DisplayClass4_0.<InvokeNext>b__0() in C:\\BuildAgent\\work\\3206e2123f54fce4\\src\\NServiceBus.Core\\Pipeline\\BehaviorChain.cs:line 95\r\n at NServiceBus.InvokeSagaNotFoundBehavior.Invoke(IncomingContext context, Action next) in C:\\BuildAgent\\work\\3206e2123f54fce4\\src\\NServiceBus.Core\\Unicast\\Behaviors\\InvokeSagaNotFoundBehavior.cs:line 17\r\n at NServiceBus.BehaviorChain
1.InvokeNext(T context) in C:\BuildAgent\work\3206e2123f54fce4\src\NServiceBus.Core\Pipeline\BehaviorChain.cs:line 107\r\n at NServiceBus.BehaviorChain1.<>c__DisplayClass4_0.<InvokeNext>b__0() in C:\\BuildAgent\\work\\3206e2123f54fce4\\src\\NServiceBus.Core\\Pipeline\\BehaviorChain.cs:line 95\r\n at NServiceBus.BehaviorChain
1.InvokeNext(T context) in C:\BuildAgent\work\3206e2123f54fce4\src\NServiceBus.Core\Pipeline\BehaviorChain.cs:line 107\r\n at NServiceBus.BehaviorChain1.<>c__DisplayClass4_0.<InvokeNext>b__0() in C:\\BuildAgent\\work\\3206e2123f54fce4\\src\\NServiceBus.Core\\Pipeline\\BehaviorChain.cs:line 95\r\n at NServiceBus.ExecuteLogicalMessagesBehavior.Invoke(IncomingContext context, Action next) in C:\\BuildAgent\\work\\3206e2123f54fce4\\src\\NServiceBus.Core\\Unicast\\Messages\\ExecuteLogicalMessagesBehavior.cs:line 24\r\n at NServiceBus.BehaviorChain
1.InvokeNext(T context) in C:\BuildAgent\work\3206e2123f54fce4\src\NServiceBus.Core\Pipeline\BehaviorChain.cs:line 107\r\n at NServiceBus.BehaviorChain1.<>c__DisplayClass4_0.<InvokeNext>b__0() in C:\\BuildAgent\\work\\3206e2123f54fce4\\src\\NServiceBus.Core\\Pipeline\\BehaviorChain.cs:line 95\r\n at NServiceBus.BehaviorChain
1.InvokeNext(T context) in C:\BuildAgent\work\3206e2123f54fce4\src\NServiceBus.Core\Pipeline\BehaviorChain.cs:line 107\r\n at NServiceBus.BehaviorChain1.<>c__DisplayClass4_0.<InvokeNext>b__0() in C:\\BuildAgent\\work\\3206e2123f54fce4\\src\\NServiceBus.Core\\Pipeline\\BehaviorChain.cs:line 95\r\n at NServiceBus.BehaviorChain
1.InvokeNext(T context) in C:\BuildAgent\work\3206e2123f54fce4\src\NServiceBus.Core\Pipeline\BehaviorChain.cs:line 107\r\n at NServiceBus.BehaviorChain1.<>c__DisplayClass4_0.<InvokeNext>b__0() in C:\\BuildAgent\\work\\3206e2123f54fce4\\src\\NServiceBus.Core\\Pipeline\\BehaviorChain.cs:line 95\r\n at NServiceBus.BehaviorChain
1.InvokeNext(T context) in C:\BuildAgent\work\3206e2123f54fce4\src\NServiceBus.Core\Pipeline\BehaviorChain.cs:line 107\r\n at NServiceBus.BehaviorChain1.<>c__DisplayClass4_0.<InvokeNext>b__0() in C:\\BuildAgent\\work\\3206e2123f54fce4\\src\\NServiceBus.Core\\Pipeline\\BehaviorChain.cs:line 95\r\n at NServiceBus.Persistence.NHibernate.OpenSessionBehavior.InnerInvoke(BehaviorContext context, Action next, Func
1 connectionRetriever) in C:\BuildAgent\work\5135de308b2f3016\src\NServiceBus.NHibernate\SharedSession\OpenSessionBehavior.cs:line 83\r\n at NServiceBus.Persistence.NHibernate.OpenSessionBehavior.Invoke(IncomingContext context, Action next) in C:\BuildAgent\work\5135de308b2f3016\src\NServiceBus.NHibernate\SharedSession\OpenSessionBehavior.cs:line 58\r\n at NServiceBus.BehaviorChain1.InvokeNext(T context) in C:\\BuildAgent\\work\\3206e2123f54fce4\\src\\NServiceBus.Core\\Pipeline\\BehaviorChain.cs:line 107\r\n at NServiceBus.BehaviorChain
1.<>c__DisplayClass4_0.b__0() in C:\BuildAgent\work\3206e2123f54fce4\src\NServiceBus.Core\Pipeline\BehaviorChain.cs:line 95\r\n at NServiceBus.BehaviorChain1.InvokeNext(T context) in C:\\BuildAgent\\work\\3206e2123f54fce4\\src\\NServiceBus.Core\\Pipeline\\BehaviorChain.cs:line 107\r\n at NServiceBus.BehaviorChain
1.<>c__DisplayClass4_0.b__0() in C:\BuildAgent\work\3206e2123f54fce4\src\NServiceBus.Core\Pipeline\BehaviorChain.cs:line 95\r\n at NServiceBus.UnitOfWorkBehavior.Invoke(IncomingContext context, Action next) in C:\BuildAgent\work\3206e2123f54fce4\src\NServiceBus.Core\UnitOfWork\UnitOfWorkBehavior.cs:line 42\r\n at NServiceBus.BehaviorChain1.InvokeNext(T context) in C:\\BuildAgent\\work\\3206e2123f54fce4\\src\\NServiceBus.Core\\Pipeline\\BehaviorChain.cs:line 107\r\n at NServiceBus.BehaviorChain
1.<>c__DisplayClass4_0.b__0() in C:\BuildAgent\work\3206e2123f54fce4\src\NServiceBus.Core\Pipeline\BehaviorChain.cs:line 95\r\n at NServiceBus.Persistence.NHibernate.OpenSqlConnectionBehavior.Invoke(IncomingContext context, Action next) in C:\BuildAgent\work\5135de308b2f3016\src\NServiceBus.NHibernate\SharedSession\OpenSqlConnectionBehavior.cs:line 43\r\n at NServiceBus.BehaviorChain1.InvokeNext(T context) in C:\\BuildAgent\\work\\3206e2123f54fce4\\src\\NServiceBus.Core\\Pipeline\\BehaviorChain.cs:line 107\r\n at NServiceBus.BehaviorChain
1.<>c__DisplayClass4_0.b__0() in C:\BuildAgent\work\3206e2123f54fce4\src\NServiceBus.Core\Pipeline\BehaviorChain.cs:line 95\r\n at NServiceBus.ChildContainerBehavior.Invoke(IncomingContext context, Action next) in C:\BuildAgent\work\3206e2123f54fce4\src\NServiceBus.Core\Unicast\Behaviors\ChildContainerBehavior.cs:line 17\r\n at NServiceBus.BehaviorChain1.InvokeNext(T context) in C:\\BuildAgent\\work\\3206e2123f54fce4\\src\\NServiceBus.Core\\Pipeline\\BehaviorChain.cs:line 107\r\n at NServiceBus.BehaviorChain
1.<>c__DisplayClass4_0.b__0() in C:\BuildAgent\work\3206e2123f54fce4\src\NServiceBus.Core\Pipeline\BehaviorChain.cs:line 95\r\n at NServiceBus.ProcessingStatisticsBehavior.Invoke(IncomingContext context, Action next) in C:\BuildAgent\work\3206e2123f54fce4\src\NServiceBus.Core\Monitoring\ProcessingStatisticsBehavior.cs:line 23\r\n at NServiceBus.BehaviorChain1.InvokeNext(T context) in C:\\BuildAgent\\work\\3206e2123f54fce4\\src\\NServiceBus.Core\\Pipeline\\BehaviorChain.cs:line 107\r\n at NServiceBus.BehaviorChain
1.<>c__DisplayClass4_0.b__0() in C:\BuildAgent\work\3206e2123f54fce4\src\NServiceBus.Core\Pipeline\BehaviorChain.cs:line 95\r\n at NServiceBus.AuditBehavior.Invoke(IncomingContext context, Action next) in C:\BuildAgent\work\3206e2123f54fce4\src\NServiceBus.Core\Audit\AuditBehavior.cs:line 20\r\n at NServiceBus.BehaviorChain1.InvokeNext(T context) in C:\\BuildAgent\\work\\3206e2123f54fce4\\src\\NServiceBus.Core\\Pipeline\\BehaviorChain.cs:line 107\r\n at NServiceBus.BehaviorChain
1.Invoke() in C:\BuildAgent\work\3206e2123f54fce4\src\NServiceBus.Core\Pipeline\BehaviorChain.cs:line 52\r\n at NServiceBus.Pipeline.PipelineExecutor.Execute[T](BehaviorChain`1 pipelineAction, T context) in C:\BuildAgent\work\3206e2123f54fce4\src\NServiceBus.Core\Pipeline\PipelineExecutor.cs:line 129\r\n at NServiceBus.Unicast.Transport.TransportReceiver.OnTransportMessageReceived(TransportMessage msg) in C:\BuildAgent\work\3206e2123f54fce4\src\NServiceBus.Core\Unicast\Transport\TransportReceiver.cs:line 413\r\n at NServiceBus.Unicast.Transport.TransportReceiver.ProcessMessage(TransportMessage message) in C:\BuildAgent\work\3206e2123f54fce4\src\NServiceBus.Core\Unicast\Transport\TransportReceiver.cs:line 344\r\n at NServiceBus.Unicast.Transport.TransportReceiver.TryProcess(TransportMessage message) in C:\BuildAgent\work\3206e2123f54fce4\src\NServiceBus.Core\Unicast\Transport\TransportReceiver.cs:line 230\r\n at NServiceBus.Transports.SQLServer.AmbientTransactionReceiveStrategy.TryReceiveFrom(TableBasedQueue queue) in C:\BuildAgent\work\888dd942b44bad5a\src\NServiceBus.SqlServer\AmbientTransactionReceiveStrategy.cs:line 57
“NServiceBus.FailedQ”:"ABC.Endpoints.XYZ
“NServiceBus.TimeOfFailure”:"2018-08-03 10:20:30:223725 Z
“NServiceBus.Retries.Timestamp”:"2018-08-03 10:19:12:234333 Z
“NServiceBus.Timeout.RouteExpiredTimeoutTo”:"ABC.Endpoints.XYZ
“NServiceBus.Timeout.Expire”:"2018-08-03 10:20:22:370495 Z
“NServiceBus.RelatedToTimeoutId”:“daad1b1d-06bf-4da7-907b-a9310104efe6”}
--------------------------------ServiceControlLog------------------------------------
16:42:00.0939|527|Warn|ServiceControl.Recoverability.ReturnToSenderDequeuer|69b815a8-bf4f-48da-beef-a93101048cff: Error forwarding message, resetting headers
2018-08-03 16:42:00.0939|527|Warn|ServiceControl.Recoverability.ReturnToSenderDequeuer|69b815a8-bf4f-48da-beef-a93101048cff: Error forwarding message, resetting headers
2018-08-03 16:42:00.0939|527|Warn|ServiceControl.Recoverability.ReturnToSenderDequeuer|69b815a8-bf4f-48da-beef-a93101048cff: Error forwarding message, resetting headers
2018-08-03 16:42:00.0939|527|Warn|ServiceControl.Recoverability.ReturnToSenderDequeuer+CaptureIfMessageSendingFails|Failed to send ‘06683242-a016-5871-b5dd-eafbe622096f’ message to ‘ABC.Endpoints.XYZ’ for retry. Attempting to revert message status to unresolved so it can be tried again.
NServiceBus.Unicast.Queuing.QueueNotFoundException: Failed to send message to address: [ABC.Endpoints.XYZ] —> System.Data.SqlClient.SqlException: Invalid object name ‘dbo.ABC.Endpoints.XYZ’.
at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action1 wrapCloseInAction) at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose) at System.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady) at System.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString, Boolean isInternal, Boolean forDescribeParameterEncryption) at System.Data.SqlClient.SqlCommand.RunExecuteReaderTds(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, Boolean async, Int32 timeout, Task& task, Boolean asyncWrite, Boolean inRetry, SqlDataReader ds, Boolean describeParameterEncryptionRequest) at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method, TaskCompletionSource
1 completion, Int32 timeout, Task& task, Boolean& usedCache, Boolean asyncWrite, Boolean inRetry)
at System.Data.SqlClient.SqlCommand.InternalExecuteNonQuery(TaskCompletionSource`1 completion, String methodName, Boolean sendToPipe, Int32 timeout, Boolean& usedCache, Boolean asyncWrite, Boolean inRetry)
at System.Data.SqlClient.SqlCommand.ExecuteNonQuery()
at NServiceBus.Transports.SQLServer.TableBasedQueue.Send(Object[] messageData, SqlConnection connection, SqlTransaction transaction) in C:\BuildAgent\work\888dd942b44bad5a\src\NServiceBus.SqlServer\TableBasedQueue.cs:line 45
at NServiceBus.Transports.SQLServer.SqlServerMessageSender.Send(TransportMessage message, SendOptions sendOptions) in C:\BuildAgent\work\888dd942b44bad5a\src\NServiceBus.SqlServer\SqlServerMessageSender.cs:line 38
— End of inner exception stack trace —
at NServiceBus.Transports.SQLServer.SqlServerMessageSender.ThrowQueueNotFoundException(Address destination, SqlException ex) in C:\BuildAgent\work\888dd942b44bad5a\src\NServiceBus.SqlServer\SqlServerMessageSender.cs:line 91
at NServiceBus.Transports.SQLServer.SqlServerMessageSender.Send(TransportMessage message, SendOptions sendOptions) in C:\BuildAgent\work\888dd942b44bad5a\src\NServiceBus.SqlServer\SqlServerMessageSender.cs:line 77
at ServiceControl.Recoverability.ReturnToSenderDequeuer.HandleMessage(TransportMessage message, IBodyStorage bodyStorage, ISendMessages sender) in C:\BuildAgent\work\7189a56f9f44affc\src\ServiceControl\Recoverability\Retrying\Infrastructure\ReturnToSenderDequeuer.cs:line 159
at ServiceControl.Recoverability.ReturnToSenderDequeuer.Handle(TransportMessage message) in C:\BuildAgent\work\7189a56f9f44affc\src\ServiceControl\Recoverability\Retrying\Infrastructure\ReturnToSenderDequeuer.cs:line 62
at NServiceBus.Satellites.SatelliteLauncher.HandleMessageReceived(Object sender, TransportMessageReceivedEventArgs e, ISatellite satellite) in C:\BuildAgent\work\a93f853f0c1b9532\src\NServiceBus.Core\Satellites\SatelliteLauncher.cs:line 89
at NServiceBus.Unicast.Transport.TransportReceiver.OnTransportMessageReceived(TransportMessage msg) in C:\BuildAgent\work\a93f853f0c1b9532\src\NServiceBus.Core\Unicast\Transport\TransportReceiver.cs:line 413
at NServiceBus.Unicast.Transport.TransportReceiver.ProcessMessage(TransportMessage message) in C:\BuildAgent\work\a93f853f0c1b9532\src\NServiceBus.Core\Unicast\Transport\TransportReceiver.cs:line 344
at NServiceBus.Unicast.Transport.TransportReceiver.TryProcess(TransportMessage message) in C:\BuildAgent\work\a93f853f0c1b9532\src\NServiceBus.Core\Unicast\Transport\TransportReceiver.cs:line 230
at NServiceBus.Transports.SQLServer.NativeTransactionReceiveStrategy.TryReceiveFrom(TableBasedQueue queue) in C:\BuildAgent\work\888dd942b44bad5a\src\NServiceBus.SqlServer\NativeTransactionReceiveStrategy.cs:line 63