Good Morning All,
This past week we recently made an update to our Azure Storage Accounts to disable blob public access due to a recommendation from our internal security team:
This appears to have broken the Azure Storage Queues Delayed Delivery functionality because now our service is pumping out these warnings constantly:
NServiceBus.Transport.AzureStorageQueues.DelayedMessagesPoller
Failed to fetch delayed messages from the storage
"Microsoft.WindowsAzure.Storage.StorageException: The remote server returned an error: (409) Conflict. —> System.Net.WebException: The remote server returned an error: (409) Conflict.
at Microsoft.WindowsAzure.Storage.Shared.Protocol.HttpResponseParsers.ProcessExpectedStatusCodeNoException[T](HttpStatusCode expectedStatusCode, HttpStatusCode actualStatusCode, T retVal, StorageCommandBase1 cmd, Exception ex) in c:\Program Files (x86)\Jenkins\workspace\release_dotnet_master\Lib\Common\Shared\Protocol\HttpResponseParsers.Common.cs:line 50 at Microsoft.WindowsAzure.Storage.Blob.CloudBlobContainer.<>c__DisplayClass24.<CreateContainerImpl>b__23(RESTCommand1 cmd, HttpWebResponse resp, Exception ex, OperationContext ctx) in c:\Program Files (x86)\Jenkins\workspace\release_dotnet_master\Lib\ClassLibraryCommon\Blob\CloudBlobContainer.cs:line 2514
at Microsoft.WindowsAzure.Storage.Core.Executor.Executor.EndGetResponse[T](IAsyncResult getResponseResult) in c:\Program Files (x86)\Jenkins\workspace\release_dotnet_master\Lib\ClassLibraryCommon\Core\Executor\Executor.cs:line 299
— End of inner exception stack trace —
at Microsoft.WindowsAzure.Storage.Core.Executor.Executor.EndExecuteAsync[T](IAsyncResult result) in c:\Program Files (x86)\Jenkins\workspace\release_dotnet_master\Lib\ClassLibraryCommon\Core\Executor\Executor.cs:line 50
at Microsoft.WindowsAzure.Storage.Blob.CloudBlobContainer.EndCreateIfNotExists(IAsyncResult asyncResult) in c:\Program Files (x86)\Jenkins\workspace\release_dotnet_master\Lib\ClassLibraryCommon\Blob\CloudBlobContainer.cs:line 329
at Microsoft.WindowsAzure.Storage.Core.Util.AsyncExtensions.<>c__DisplayClass2`1.b__0(IAsyncResult ar) in c:\Program Files (x86)\Jenkins\workspace\release_dotnet_master\Lib\ClassLibraryCommon\Core\Util\AsyncExtensions.cs:line 69
— End of stack trace from previous location where exception was thrown —
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at NServiceBus.Transport.AzureStorageQueues.LockManager.d__7.MoveNext() in C:\BuildAgent\work\3c19e2a032c05076\src\Transport\DelayDelivery\LockManager.cs:line 75
— End of stack trace from previous location where exception was thrown —
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at NServiceBus.Transport.AzureStorageQueues.LockManager.d__5.MoveNext() in C:\BuildAgent\work\3c19e2a032c05076\src\Transport\DelayDelivery\LockManager.cs:line 29
— End of stack trace from previous location where exception was thrown —
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at NServiceBus.Transport.AzureStorageQueues.DelayedMessagesPoller.d__16.MoveNext() in C:\BuildAgent\work\3c19e2a032c05076\src\Transport\DelayDelivery\DelayedMessagesPoller.cs:line 94
— End of stack trace from previous location where exception was thrown —
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at NServiceBus.Transport.AzureStorageQueues.DelayedMessagesPoller.d__15.MoveNext() in C:\BuildAgent\work\3c19e2a032c05076\src\Transport\DelayDelivery\DelayedMessagesPoller.cs:line 64
Request Information
RequestID:20b97765-c01e-006e-20a3-50c249000000
RequestDate:Mon, 24 May 2021 13:51:21 GMT
StatusMessage:Public access is not permitted on this storage account.
ErrorCode:PublicAccessNotPermitted
ErrorMessage:Public access is not permitted on this storage account.
RequestId:20b97765-c01e-006e-20a3-50c249000000
Time:2021-05-24T13:51:21.8756449Z
We are currently using NServiceBus v7.1.6 and ServiceBus.Azure.Transports.WindowsAzureStorageQueues v8.2.2
To me this appears to show that the ASQ delayed delivery specific storage table which is being used to store the delayed messages is being accessed internally by NSB via public authentication. Is there a way to update this via our service configuration file so that it doesn’t use public authentication (since NSB already has the storage account connection information which I would think would have the appropriate access) so we don’t get these warnings?