Azure Functions Transaction Mode Error - .NET 6.0

1

I’m working with Azure Functions, using

  • .NET 6.0
  • NServiceBus.AzureFunctions.InProcess.ServiceBus 2.1

I want to publish a message within my Azure Function, so I want to achieve TransportTransactionMode.SendsAtomicWithReceive as defined here:

However, when I set the AutoComplete to false:

ServiceBusTrigger("ProcessMessageTx", AutoComplete = false)]

my function fails with:

[2021-12-21T20:01:54.853Z] Handling XXXXCommand in XXXCommandHandler
[2021-12-21T20:01:54.966Z] Delayed Retry will reschedule message '792801d2-9913-4d04-8403-ae05014a158d' after a delay of 00:00:10 because of an exception:
[2021-12-21T20:01:54.967Z] Microsoft.Azure.ServiceBus: Local transactions are not supported with other resource managers/DTC.
[2021-12-21T20:01:56.331Z] Executed 'zp_adapt_box' (Failed, Id=627ddf2b-3ae4-460c-9d55-1897942be5fc, Duration=1492ms)
[2021-12-21T20:01:56.333Z] System.Private.CoreLib: Exception while executing function: zp_adapt_box. Microsoft.Azure.ServiceBus: Local transactions are not supported with other resource managers/DTC.

I’m not introducing any transactions as far as I can tell.

My Trigger code:


using System.Threading.Tasks;
using Microsoft.Azure.ServiceBus;
using Microsoft.Azure.ServiceBus.Core;
using Microsoft.Azure.WebJobs;
using Microsoft.Extensions.Logging;
using NServiceBus;

namespace gladaptbox;
internal class NServiceBusTrigger
{
    private readonly IFunctionEndpoint _functionEndpoint;

    public NServiceBusTrigger(IFunctionEndpoint functionEndpoint)
    {
        _functionEndpoint = functionEndpoint;
    }

    [FunctionName("XXXXXXXX")]
    public async Task Run(
        [ServiceBusTrigger("XXXXXXXX", AutoComplete = false)]
        Message message,
        ILogger logger,
        MessageReceiver messageReceiver,
        ExecutionContext executionContext)
    {
        await _functionEndpoint.Process(message, executionContext, messageReceiver, logger);
    }
}

My Startup Code:

using System;
using gladaptbox;
using Microsoft.Azure.Functions.Extensions.DependencyInjection;
using NServiceBus;

[assembly: FunctionsStartup(typeof(Startup))]
//[assembly: NServiceBusTriggerFunction(Startup.EndpointName, SendsAtomicWithReceive = true)]

namespace gladaptbox;

public class Startup : FunctionsStartup
{
    public const string EndpointName = "zp_adapt_box";
    private const string _serviceControlMetricsAddress = "particular.monitoring";

    public override void Configure(IFunctionsHostBuilder builder)
    {
        var services = builder.Services;
        builder.UseNServiceBus(
            () =>
            {
                var endpointConfiguration = new ServiceBusTriggeredEndpointConfiguration(EndpointName);
                var advancedConfiguration = endpointConfiguration.AdvancedConfiguration;

                endpointConfiguration.LogDiagnostics();

                // enable audit and error message processing
                advancedConfiguration.AuditProcessedMessagesTo("audit");
                advancedConfiguration.SendFailedMessagesTo("error");

                // enable metrics
                var metrics = advancedConfiguration.EnableMetrics();
                metrics.SendMetricDataToServiceControl(
                    serviceControlMetricsAddress: _serviceControlMetricsAddress,
                    interval: TimeSpan.FromSeconds(10));

                return endpointConfiguration;
            });
    }
}

Any thoughts on why this is happening?

Thanks in adavance - jlo

2

What does your handler do?

3

I’m working on a POC with Azure Functions. The handler has a placeholder for a call to an external API - at the moment it is doing nothing. Then it publishes an Event to alert other services. My assumption is the processing of the incoming message and the Publish can occur in an atomic transaction.


    #region IHandleMessages<DropBoxFileCommand>

    public async Task Handle(DropBoxFileCommand message, IMessageHandlerContext context)
    {
        _log.Warn($"Handling {nameof(DropBoxFileCommand)} in {nameof(BoxHandler)} file name {message.BoxFile.FileName}");

        await context.Publish(new BoxFileDroppedEvent(message.BoxFile));
    }

    #endregion
4

Hi Jeff!

I’ve confirmed that this is an issue on our end due to us still being on the old v4 functions SDK. I’ve raised an issue that goes into more detail, feel free to subscribe and chime in if needed

We’ll discuss this internally and I’ll report back here as soon as we have an ETA for the fix.

Thanks for reporting the bug!

Cheers,

Andreas

5

Thanks - subscribed to the GitHub Issue - I can move forward with my POC in the meantime. Quick response very much appreciated ;>

6

@jlo we’ve just released 3.0.0 that should address your problem. Thanks for reaching out!

1 Like
7

Thanks again for addressing this so quickly. I will update my projects and report back if I see any further issues. :smiley:

8

I continue to have issues after upgrading to NServiceBus.AzureFunctions.InProcess.ServiceBus 3.0.0.

I tested both the code generation option using:

[assembly: NServiceBusTriggerFunction(ApplicationConstants.QueueName, SendsAtomicWithReceive = true, TriggerFunctionName = ApplicationConstants.FunctionName)]

and providing my own trigger (in the attached zip file )
LocalTransactionError.zip (9.8 KB)

Also - in the attached zip file is the detailled endpoint log.

As a reminder, in the handler I publish a message and am expecting that to occur in a transaction with the message I’m handling being removed form the queue.

When I turn Atomic Transaction off the code works. When I turn it on I get the following:

[2022-01-14T16:30:56.199Z] Delayed Retry will reschedule message '4420dcab-5336-46ac-a954-aee087b03e4b' after a delay of 00:00:05 because of an exception:
[2022-01-14T16:30:56.200Z] Azure.Messaging.ServiceBus: Local transactions cannot span multiple entities or partitions.
[2022-01-14T16:30:56.650Z] Executed 'zp_adapt_box' (Succeeded, Id=f47c2628-33b3-4e8b-af38-9c690be413c4, Duration=585ms)
[2022-01-14T16:30:58.913Z] Executing 'zp_adapt_box' (Reason='(null)', Id=30a40a6a-79ad-4dfe-8cb1-6238b010a161)
nId: (null)
[2022-01-14T16:30:58.916Z] Handling DropBoxFileCommand in DropBoxFileCommandHandler
[2022-01-14T16:30:59.067Z] Immediate Retry is going to retry message 'b75f3c33-a750-471f-b1bb-86d09bdd045a' because of an exception:
[2022-01-14T16:30:59.077Z] Azure.Messaging.ServiceBus: Local transactions cannot span multiple entities or partitions.
[2022-01-14T16:30:59.148Z] Executed 'zp_adapt_box' (Succeeded, Id=30a40a6a-79ad-4dfe-8cb1-6238b010a161, Duration=235ms)
[2022-01-14T16:30:59.152Z] Executing 'zp_adapt_box' (Reason='(null)', Id=49a5638b-80ba-4fd4-aab5-88b3a95bb071)
[2022-01-14T16:30:59.154Z] Trigger Details: MessageId: 9e094f51-c2f6-47c2-a28e-98b85faeaa88, SequenceNumber: 40, DeliveryCount: 2, EnqueuedTimeUtc: 2022-01-14T16:30:58.1530000+00:00, LockedUntilUtc: 2022-01-14T16:35:58.4000000+00:00, SessionId: (null)
[2022-01-14T16:30:59.163Z] Handling DropBoxFileCommand in DropBoxFileCommandHandler
[2022-01-14T16:30:59.306Z] Moving message 'b75f3c33-a750-471f-b1bb-86d09bdd045a' to the error queue 'error' because processing failed due to an exception:
[2022-01-14T16:30:59.309Z] Azure.Messaging.ServiceBus: Local transactions cannot span multiple entities or partitions.
[2022-01-14T16:30:59.610Z] Executed 'zp_adapt_box' (Failed, Id=49a5638b-80ba-4fd4-aab5-88b3a95bb071, Duration=455ms)
[2022-01-14T16:30:59.612Z] System.Private.CoreLib: Exception while executing function: zp_adapt_box. Azure.Messaging.ServiceBus: Local transactions cannot span multiple entities or partitions.
[2022-01-14T16:30:59.618Z] Message processing error (Action=ProcessMessageCallback, EntityPath=zp_adapt_box, Endpoint=jefftest123.servicebus.windows.net)
[2022-01-14T16:30:59.620Z] System.Private.CoreLib: Exception while executing function: zp_adapt_box. Azure.Messaging.ServiceBus: Local transactions cannot span multiple entities or partitions.
[2022-01-14T16:30:59.624Z] Message processing error (Action=Abandon, EntityPath=zp_adapt_box, Endpoint=jefftest123.servicebus.windows.net)
[2022-01-14T16:30:59.625Z] Azure.Messaging.ServiceBus: The lock supplied is invalid. Either the lock expired, or the message has already been removed from the queue, or was received by a different receiver instance. (MessageLockLost).

Note it even prevents my message from moving to the error log.

I’m wondering if this needs to be set by the transport: ServiceBusClientOptions.EnableCrossEntityTransactions Property (Azure.Messaging.ServiceBus) - Azure for .NET Developers | Microsoft Docs

TIA - jlo

9

Hi!

We were able to reproduce the issue using this sample where the SendLocal would blow up before the changes made in v3

but we might have missed something.

Would you be able to try that sample and see if it works for you?

Also could you perhaps zip the entire repro and attach it here? (or email it to support@particular.net)

Cheers,

Andreas

10

I will although I am getting on a flight shortly and it will probably be Monday.

However - in your test above - you are doing a SendLocal in the handler- which by definition would be on the same queue right? I’m doing a Publish - and I believe the transport would write to a different queue. This seems to be the issue - transacting over more than one queue.

We are using some internal NuGet libs so I can’t give you our repo but can try to create a repro project after I complete travel this weekend.

11

Cross-entity transaction indicates how messages are sent. Previously called “send-via”. So it’s not just the number of entities participating in the operation but also the method of dispatching involved.
To guarantee it’s working, the test should probably have 2 entities participating (sender & receiver endpoints).

12

That is true but since the ServiceBusClient is created for us by the functions host we need to rely on MS doing this for us. We did discuss this with them when we raised the lack of support for send-via but I’ll double-check the actual implementation and report back here.

@jlo However - in your test above - you are doing a SendLocal in the handler- which by definition would be on the same queue right? I’m doing a Publish - and I believe the transport would write to a different queue. This seems to be the issue - transacting over more than one queue.

@seanfeldman To guarantee it’s working, the test should probably have 2 entities participating (sender & receiver endpoints).

We do have acceptance tests that use a different queue using a publish and they pass.

Can you both spot anything that is missing in there?

13

I think I’ve found something. Since our acceptance tests don’t use the function host emulator yet we set the flag on the ServicebusClient to true when we run our tests. Reviewing the MS fix I can see that they added a flag on the ServiceBusOptions to mimic this

we need to find a way to set that flag.

I’ve managed to reproduce this on our end so no need to send us anything @jlo

1 Like
14

@jlo I’ve been able to fix this by changing the host.json to

{
  "version": "2.0",
  "extensions": {
    "serviceBus": {
      "EnableCrossEntityTransactions": true
    }
  }
}

Can you verify that it works?

I’ll do some more testing and:

  1. Make sure to update our samples and docs as needed
  2. See if we somehow can’t set this automatically for you to avoid this issue going forwards
15

@andreasohlund that makes sense if options are created when Service Bus client is created and registered by Functions. EnableCrossEntityTransactions option appears to be already documented for host.json.