I have an annoying issue. I’m using msmq persistence and transport. This in it self is fine and works almost painlessly.
The issue is this. When subscribing to events, the subscription is placed in the proper msmq queue, but a new subscription is made every time I restart my application. I’ve tried switching to MS SQL persistence and here it works fine. When restarting my application the “old” subscription is continued and no duplicates are present.
The question is, am I missing something since there are duplicates/triplicates/… and so on?
Are you using containers/docker by any mean? Can you have a look at the subscription storage and post here the details about the duplicate subscriptions? Can you spot any obvious difference in them?
No dockers or containers.
The solution is being rebuild, and I’ve noticed no changes in the messages, but maybe there is a change in the identifier for the message sender. I’ll try to perform tests without rebuilding and see if it’s the same.
The funny thing about it is that with the same procedures (build and run) the SQL subscriptions don’t result in the same issue.
The code shown is for my publish only endpoint. To avoid cluttering and hold ups on my command endpoint I’m using a separate one for all publish events so I can push those messages without delays.
I do have another issue (Sending messages with a custom TimeToBeReceived is not supported on transactional MSMQ) but that might be for another thread…
public static async Task<IEndpointInstance> CreateMsmqBackendPublishEndpointInstance(NServiceBusBackendPublishEndpointTypeEnum endpointType, Func<ICriticalErrorContext, Task> onCriticalError)
{
var endpointConfiguration = new EndpointConfiguration(endpointType.ToString());
var conventions = endpointConfiguration.Conventions();
conventions.DefiningTimeToBeReceivedAs(type => TimeSpan.FromMinutes(5));
endpointConfiguration.DefineCriticalErrorAction(onCriticalError);
endpointConfiguration.EnableInstallers();
endpointConfiguration.MakeInstanceUniquelyAddressable(endpointType.ToString());
endpointConfiguration.PurgeOnStartup(true);
var recoverability = endpointConfiguration.Recoverability();
recoverability.Delayed(d => { d.NumberOfRetries(0); });
recoverability.Immediate(i => { i.NumberOfRetries(0); });
endpointConfiguration.SendFailedMessagesTo($"{endpointType.ToString()}.Errors");
endpointConfiguration.UsePersistence<InMemoryPersistence, StorageType.Timeouts>();
endpointConfiguration.UsePersistence<MsmqPersistence, StorageType.Subscriptions>()
.SubscriptionQueue($"{endpointType.ToString()}.Subscriptions");
//var persistence = endpointConfiguration.UsePersistence<SqlPersistence>();
//var subscriptions = persistence.SubscriptionSettings();
//subscriptions.CacheFor(TimeSpan.FromMinutes(1));
//var dialect = persistence.SqlDialect<SqlDialect.MsSqlServer>();
//dialect.Schema("NServiceBus");
//persistence.ConnectionBuilder(() => new SqlConnection(@"Data Source=.;Initial Catalog=;Persist Security Info=True;User ID=;Password="));
endpointConfiguration.UseSerialization<NewtonsoftSerializer>();
var transport = endpointConfiguration.UseTransport<MsmqTransport>();
transport.Transactions(TransportTransactionMode.None);
return await Endpoint.Start(endpointConfiguration);
}
So I call this method from my services to create an endpoint for publishing events, like Masterdata item upsert/deletion.
The commented out section is the one I tested with SQL that doesn’t create duplicate subscriptions.
I’m not sure I get the issue. Because initially I thought it worked correctly, until you mentioned that SQL Transport doesn’t show the behavior.
How subscriptions work in MSMQ is that every single time an endpoint is started, it figures out if it should be subscribed to certain events. If so, it will send a message to the appropriate endpoint to subscribe to those events. The reason it does this every single time, is because it has no idea if it ever subscribed before. So this is the expected behavior.
The receiving endpoint of those messages, need to store the subscription somehow. With MSMQ and SQL Transport you need persistence. With SQL Transport this is probably an easy choice. MSMQ has MSMQ Subscriptions, InMemory or SQL Persistence. Each subscription should only be stored once, obviously. And I get the impression that with MSMQ, the subscriptions is stored multiple times, in a different way? This is what isn’t clear to me. Perhaps you could share what exactly is stored, if you can find out.
What might have something to do with it is this line:
You can use that, but it’s not likely you should use an enum to differentiate between them. It’s unclear what options there are in the enum and what is provided, but usually, you’d use a number of GUID in there. It’s used for callbacks and sender-side-distribution. Do you make use of any of these? If not, you should probably remove that line.
Also endpointConfiguration.PurgeOnStartup(true); should probably be removed, since it will immediately remove the subscription messages that other endpoints might have sent.
I would expect that the subscription should be picked up by either the source computer GUID and Name are the same as are the label. It would make sense that those identifiers (since they are the same) would make sure that only one subscription is in the respective queue for the respective client just once. This is where I get the difference in results for the MsmqPersistence and the SqlPersistence.
On the client side I use an InMemoryPersistence to store the subscription data.
The Enum I put in there is used to identify the individual service and client so I always address the same way and don’t get typos. This is also working when using SqlPersistence.
This have been put there for development purposes only.
Again, what I don’t get is why there is a difference between the Msmq and the Sql implementation. The Sql can “pick up” on existing subscriptions by matching the client name. How come the Msmq can’t?
MSMQ Persistence if part of the MSMQ Transport package where SQL Persistence is a very different package then the SQL Transport.
MSMQ Persistence assumes that the subscriptions are stored locally. If an endpoint is a publisher, this instance would always receive a subscription message, updates it in memory state, and writes it to the subscription queue. This queue can contain duplicates
When restarting the publisher, all subscriptions will be read, deduplicated, and written back.
It is the publisher that reads/writes from configure storage provider. The subscriber sends a subscription message to the publisher queue.
With MSMQ this means the publisher is writing an subscription entry to the subscription queue and not the subscriber. WHen it needs to Publish the subscription entry is already in memory and does not need to be read from the subscription queue.
The way SQL persistence works is that all instances do not have their own local storage but have a shared storage location, instead of an in-memory state it uses caching to not having to query the subscription location for every publish.
I was typing a response when @ramonsmits also provided information.
MSMQ deduplicates subscriptions on startup. You can see that right here in our code.
For safety it stores the subscription. So there might be duplicates in MSMQ. However you should never receive a published event twice on the same endpoint, because of this duplication.
Can you confirm this is true, or do you actually receive duplicate events?
I get what you say, but maybe I’m missing the point where it answers my question.
For Msmq I see this:
My publish backend service is running. No subscriptions in the queue
I start my frontend application and it subscribes. There now one “set” of subscriptions as the services might subscribe to several events.
I close my frontend application.
I start up the frontend application again. Now I have 2 “sets” of subscriptions as a new set is made every time the frontend starts up and subscribes to events.
repeat step 3 + 4 and a new set is added every time and over time that will take up space as they are not removed again.
For SQL I see this:
My publish backend service is running. No subscriptions in the queue
I start my frontend application and it subscribes. There now one “set” of subscriptions as the services might subscribe to several events.
I close my frontend application.
I start up the frontend application again. Now I still only have one set of subscriptions.
repeat step 3 + 4 and I still only have one set.
I don’t understand where you think I’m confused about the difference between transport and persistence. I get the difference and I think I’m explaining the exact same issue in my replies. I’m now trying a new approach to explain my issue.
I will test if I actually get more than one publish to multiple subscriptions. But that’s not even the point I was getting at. I’m aiming at the fact that over time the queue will fill up ever so slowly.
The difference between SQL and MSMQ transports is that the first is a broker type message queue and MSMQ is a federated one. More details here.
As Ramon explained and in the lines of code I linked, the subscriptions definitely should be deduplicated.
So it can add up to a few messages, but they should be deduplicated. I just tested it with v6 and v7 of NServiceBus and this is exactly what I’m seeing. The reason we’re creating duplicates at some point is to be 100% sure we don’t lose any of the subscriptions. Rather duplicates, than accidental misses. Duplicates in storage, not when publishing actual messages though.
Okay, I will keep an eye on it. I got the same result as you on receiving multiple messages for the duplicated subscriptions, only one publish even with more subscriptions to the same event for the same client.
Thank you guys for beating some understanding into my rookie head.
