I’m finishing up migrating my application from NServiceBus 5 to 6, and the last problem I’ve run into is with callbacks. In particular, I’m not sure what value I should be using for MakeInstanceUniquelyAddressable. The documentation explicitly mentions not to hard code the value passed into MakeInstanceUniquelyAddressable, but all of the samples in the documentation hard code the value to “1”.
I’m in a load-balanced scenario with IIS, where I have a number of identical servers that all host endpoint A inside IIS. The automated deploy process that I use sets up all of the machines with identical versions of the code and configuration files. The load balancer arbitrarily chooses one of the servers to handle the request. The IIS-hosted endpoint uses Bus.Request to send a command to an endpoint hosted on some other server and wait for the response code.
In this scenario, does the value passed to MakeInstanceUniquelyAddressable need to be globally unique across all of the load-balanced servers, or does the value just need to be unique inside each individual server instance? If they need to be globally unique, how do you recommend dealing with this when you have a number of otherwise identical servers?
Hi Dennis,
Are you using MSMQ with private queues?
Regards
Daniel
Yes, I’m using MSMQ, and each server has its own set of queues that are used with the endpoint. The queue names are all the same, though obviously the machine names are different for each one. For example, I would have myendpoint@server1, myendpoint@server2, etc.
Hi Dennis
Yeah, when we moved out the callbacks package we haven’t considered this scenario. When we initially designed the new callback component, we talked about the IIS recycling scenario. In recycling scenarios, it is possible that you have two AppDomain for the same endpoint running at the same time in the same IIS and those the receives would become competing consumers. In such a scenario you’d potentially lose the reply because the reply could be consumed by the instance that does not have the InMemory callback registered.
But still since this is a really unlikely scenario I guess we should not enforce unique IDs and allow you to opt-out from that. I will raise an issue and keep you posted.
Regards
Daniel
Is there something I can do as a workaround in the meantime? This is a very infrequently used portion of my application, so I’m willing to risk losing the reply during IIS recycling for the time being if I can get it working in the general case.
Hi Dennis
You could custom build the callbacks and use that version.
Checkout master branch, comment and NServiceBus.Callbacks/CallbackRequestSupport.cs at master · Particular/NServiceBus.Callbacks · GitHub and create a release build. Then you should have a NuGet package in the folder that you could add to your NuGet feed or temporary check that packet into source control. Would that work for you?
Sorry that we caused you this inconvenience
Daniel
I can try building that version, but before I do that, would it actually work correctly in my situation if I actually specify a discriminator when configuring the endpoint in IIS? The caveat is that because my servers have identical configuration, they would also have the same discriminator value. If that’s not going to work, I can build the package, but I just want to make sure there is not an easier answer before I do that.
Hi!
EDIT: Disregard this post, the 2.1.0 release was abandoned. See later posts in this thread
There is now a 2.1.0 callbacks package on our MyGet feed
https://www.myget.org/feed/particular/package/nuget/NServiceBus.Callbacks/2.1.0
would you be able to verify that all works for you by using the added opt-out?
.EnableCallbacks(enforceUniqueEndpointInstanceAddress: false)
Let me know if you have any issues!
Cheers,
Andreas
Hi Andreas,
I tried using the packages you linked to, but I still get an exception when I try to send the request.
Here is what I’m doing in my code:
MyEnum result = await endpoint.Request<MyEnum>(message, cancellationTokenSource.Token);
Here is the stack trace from the exception:
at NServiceBus.ApplyReplyToAddressBehavior.Invoke(IOutgoingLogicalMessageContext context, Func`2 next) in C:\BuildAgent\work\a93f853f0c1b9532\src\NServiceBus.Core\Routing\ApplyReplyToAddressBehavior.cs:line 32
at lambda_method(Closure , IOutgoingLogicalMessageContext )
at NServiceBus.Pipeline.Behavior`1.<>c__DisplayClass0_0.<Invoke>b__0() in C:\BuildAgent\work\a93f853f0c1b9532\src\NServiceBus.Core\Pipeline\Behavior.cs:line 21
at NServiceBus.SetCallbackResponseReturnCodeBehavior.Invoke(IOutgoingLogicalMessageContext context, Func`1 next) in C:\BuildAgent\work\c907d2b57aa761fb\src\NServiceBus.Callbacks\Reply\SetCallbackResponseReturnCodeBehavior.cs:line 19
at NServiceBus.Pipeline.Behavior`1.Invoke(TContext context, Func`2 next) in C:\BuildAgent\work\a93f853f0c1b9532\src\NServiceBus.Core\Pipeline\Behavior.cs:line 21
at lambda_method(Closure , IOutgoingLogicalMessageContext )
at NServiceBus.DataBusSendBehavior.<Invoke>d__1.MoveNext() in C:\BuildAgent\work\a93f853f0c1b9532\src\NServiceBus.Core\DataBus\DataBusSendBehavior.cs:line 81
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at NServiceBus.UnicastSendRouterConnector.<Invoke>d__1.MoveNext() in C:\BuildAgent\work\a93f853f0c1b9532\src\NServiceBus.Core\Routing\Routers\UnicastSendRouterConnector.cs:line 23
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at NServiceBus.RequestResponseExtensions.<Request>d__3`1.MoveNext() in C:\BuildAgent\work\c907d2b57aa761fb\src\NServiceBus.Callbacks\RequestResponseExtensions.cs:line 110
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
I’ve omitted the rest of the code below this is the stack trace as it all comes from my application/IIS.
We’ve found the issue, I will let you know when we have an updated version
that you can test.
Thanks for your help Dennis!
Hi, again Dennis!
Please accept my apologies; it turns out that we’ve sent you on a wild goose chase. Let me explain.
As we dug deeper into this issue, it turns out the routing support in NServiceBus.Core doesn’t support the flexibility we need to make this work seamlessly for you. While that could be done via us releasing a more powerful routing model it made us realize that the instance unique queues is a better model for callbacks.
We recommend that you go ahead and configure your front ends to be uniquely addressable and “hard code” the value of something like “callbacks” or “replies” (we’ll update documentation accordingly). This way all your non-business critical messages that trigger callbacks will arrive on that queue and not risk delaying other more critical messages arriving at the main input queue. It would also allow things like purging that specific queue on IIS recycles and machine restarts if stale messages were an issue.
This does mean that you will now have one additional queue per instance to manage from an infrastructure perspective, but we believe that is a small price to pay.
Does this make any sense?
Again, sorry for wasting your time!
Cheers,
Andreas
Hi Andreas,
I configured my IIS endpoint to be uniquely addressable using “callbacks”, and everything appears to work correctly.
Thanks for all of your help.
Dennis
Pleased to hear that Dennis!
We’ve now updated the documentation to reflect this discussion
Client-side callbacks • Callbacks • Particular Docs (you might have to refresh the page)
Cheers and good luck!
Andreas