I’m trying something but a) don’t know if it’s supposed to be supported by NServiceBus. b) if yes then I’m wondering how I can solve the following exception I’m seeing. Everything compiles perfectly fine but it’s during runtime in the receiver that things go boom.
Some interface marker (I’m having a handler on that as well for the moment in my receiver):
public class SubDomainCreated<T> : IEvent, IAudit
{
public DateTime OccuranceDate { get; set; } = DateTime.UtcNow;
public DateTime RegistrationTime { get; set; } = DateTime.UtcNow;
public Guid CorrelationId { get; set; }
public Guid EventId { get; set; } = Guid.NewGuid();
public Guid ProcessId { get; set; } = Guid.NewGuid();
public T NewValue { get; set; }
public Origin Origin { get; set; }
}
public class PersonCreatedHandlerTest : IHandleMessages<SubDomainCreated<Q_Persoon>>
{
private static readonly ILog Log = LogManager.GetLogger(MethodBase.GetCurrentMethod().DeclaringType);
public Task Handle(SubDomainCreated<Q_Persoon> message, IMessageHandlerContext context)
{
Log.Info("Q_Persoon arrived with templated message");
return Task.CompletedTask;
}
}
Is the above possible? I’m getting the following error in the receiver:
ERROR AutoSubscribe was unable to subscribe to event ‘EventCatalog.SubDomainCreated1[[EventCatalog.DB2Contracts.Q_Persoon, EventCatalog, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null]]': No publisher address could be found for message type 'EventCatalog.SubDomainCreated1[EventCatalog.DB2Contracts.Q_Persoon]’. Ensure that a publisher has been configured for the event type and that the configured publisher endpoint has at least one known instance.
Update: I first let it work with the learningtransport and there it didn’t give an error but when trying it with SQL Server (message and persistence) then I get the above. Serialization problem?
as updated in my post. With learningtransport it worked out but in our application we’re currently using SQL Server for both transport and persistence.
I’m currently not at work but I know we’re not on the latest versions but on those of half a year ago. NSB version is 7.x but for sql I think it’s still version 4.x (saw some update of a component to v5.x but didn’t upgrade yet as we have a demo for the CIO on Monday and saw some things that had become obsolete so rolled back the nuget package version.
In version 4 and below, this feature relies on message-driven pub-sub which requires a separate persistence for storage of subscription information. In version 5 and above, the transport handles subscription information natively and a separate persistence is not required.
If you cannot update to version 5 of the transport and need to resolve the issue you’re having with pub/sub in version 4, you’ll need to define the publishers. Below are a few links to get you going.
We can upgrade to a newer version if needed. Don’t think it’ll take much convincing of the product owner if we can save time for implementation. We’re going to upgrade to .NET Core 3.1 in a month as well with all of the applications (50 working on the big project of which this part is just a tiny but important part).
Basically what we’re trying to do is get something consistent for OldValue and NewValue of a specific type, hence the . OldValue would be used in update scenario’s as we have to either have an update and first want to check in the receiver’s database if we’re having the correct record to update with NewValue of type T.
If there’s a better way please let us know.
CorrelationId is going to be a guid originating from a web client and being passed around many other data stores to see if all the information was spread correctly. I’m just adding it here as well as it might be of interest in the future to see where information originated from as there will be 3 or 4 different input applications that will generate information which we need to bus around to 40 receiving applications.
@Kris_van_der_Mast How can this exist? Is this some multi-master system where different systems can both modify the same data stored on different (master) locations?
How are you going to deal with conflicts?
If this is CRUD, maybe consider not using messaging for that but go directly to the data store.
If this is replication, consider using native storage replication options if available. Building your own replication logic on top of messaging is usually considered a bad practice.
If this is 2 way replication than usually this operates in a “last writer wins” or an append only model. If this is “multi master” could it happen that the same record is modified at both sides at the same time? How do you deal with such conflicts?
If this is a “one-way” replication (primary-> secondary) you could use such a type of optimistic concurrency control to maybe make sure that updates happen in the right order as messages should be assumed to to be processed out-of-order.
yes it’s possible. Thing is that the original application saw day of light in the 80’s. Loads of other applications were written on top, besides, underneath, … and sometimes legacy on legacy on legacy. A big part is also on DB2 with Cobol stored procedures with heaps of logic into them.
A team did intensive investigations about replication and found out that simple tooling, even from big known vendors, couldn’t cope with it.
Besides an initial load, basic insert over the wire to receivers, there will also be a CDC via IBM mq which will hold every single update/change/insert/… on DB2 and we need to capture that and spread out that information to well over 40 other applications. Hence the pub sub architecture via NServiceBus as the preferred solution.
Eventually we’ll also get information from other systems, again via NSB back into the DB2 database. Is this an ideal situation? Far from it but it’s what has to be done.
I didn’t express myself correctly. I didn’t mean how it can exists but how you are dealing with conflicts.
In your system you are describing replication of data between different types of data stores which is indeed not ideal but as long as data fragments are replicated only in one direction to avoid conflicts then it can work.
In such cases NServiceBus has a the benefit in that it is just c# code and you can do any data mapping and transformation as you see fit.
We’re very well aware of potential conflicts and try to find ways to be prepared for them by first checking if the OldValue was correctly there before we update with new data.
There will be a load of transformation going on as some systems need xml, others excel files, others API’s, others some funky db schema, …