This discussion topic serves as a reference point to capture challenges with the migration to the topic-per-event topology. Please share with us the challenges you faced or rough edges you discovered during the migration.
There is also a github issue you can use should you prefer this avenue.
For cases when specific data or insights of the project needs to be disclosed, feel free to raise a support request.
Other question:
Itās not clear to me from the manual. if we need todo this manual or if the installer takes care of this. I assume the later? topology.SubscribeTo<IOrderStatusChanged>("Shipping.OrderAccepted");
We have not yet made the decision when weāll upgrade the Functions support. We expect to assess its priority next week.
Itās not clear to me from the manual. if we need todo this manual or if the installer takes care of this. I assume the later?
The topology.SubscribeTo<>() API is used to override the default mapping, which is the eventās full name. In this case if you have a handler for Shipping.IOrderStatusChanged like this:
public class OrderStatusChangedHandler : IHandleMessages<IOrderStatusChanged>
{
}
and auto-subscribe enabled (default), the automatic subscription mechanism will create topic shipping.iorderstatuschanged and a subscription to that topic with forwarding to its input queue. This topic, however, will never receive any messages because publisher always publishes concrete events e.g. OrderAccepted. That events, by default, will be published to shipping.orderaccepted topic.
This why the mapping via topology.SubscribeTo is necessary in this case.
Wanted to give you a heads-up that we are making good progress with the function integration, and we are looking at an ETA of a week(ish) until it is completed if nothing else pops up during our testing.
@SzymonPobiega thanks. I rolled it out today and I will monitor it. I am happy with this change because we have hit the 5Gb limit in the past :-).
When not using interfaces, but concrete classes, do I also need to specify it explicitly?
I have the following msgs:
public record InteractionEvent : IEvent
public record InteractionSent : InteractionEvent
public record InteractionOpened : InteractionEvent
public record InteractionClicked : InteractionEvent
I only published the events derived from InteractionEvent, never the InteractionEvent itself.
I subscribe only to InteractionEvent (for now) in my handler.
I see that only the interactionevent topic is auto created (but not the derived msg types). Will this work ok, or do I need to specify explicit to publish all derived msg to interactionevent?
You need a bit of mapping here. This can be done in two ways.
Subscriber-side
On the subscriber side you can inform the endpoint that, although the type accepted by the handler is InteractionEvent, the actual topics you are interested in are named after the derived types:
With your explanation, it finally clicked for me :-).
Only concrete classes work, inheritance or any interfaces will not auto work, and you need to wire them up yourself. I think adding an example like your explanation to the manual would make it a little bit clearer (the manual doesnāt make it clear enough what does work automatically and what doesnāt).
A third option would be then to not subscribe the handler to the base class, but all the derived classes. Then I donāt need to do the manual wiring (my preference I think).
While youāre at that, it would be valuable to provide a sample of the mapping via configuration, briefly mentioned in the upgrade guide, under more advanced scenarios, but not in the topology document.
I think there is definitely room for improvement on the documentation page. I have already noted down once we have the two other PRs merged, it might be beneficial to go over the pages again to consolidate things a bit.
Hi, Iāve been doing some preliminary investigation into how our migration process will look. We have a reasonably complex landscape with a decent bit of cases of events passing in-between services (or teams), so weāll definitely have to make use of the migration topology as we move towards the upgrade.
I wanted to check a couple of conclusions/points from my investigation with you directly:
Is there any documentation on configuring the migration topology with NServiceBus on Azure Function app?
After looking at the IHostbuilder extension implementation I figured out a working setup for my spike by setting specific paths in configuration to make it create the topology as Migration topology and then modifying that topology through code later, but Iām not sure if there are better ways to configure this.
Is there any support for configuring events to āintegrateā with both the new and old topology at the same time?
Given that in many cases publisher and subscriber of our events are separate applications (often managed by different teams), getting all of their upgrade deployments synced would be quite hard.
Preferably we would use a configuration setup where we could mark an event as either to be published on both topics (so listeners on either can pickup the event), or to mark an event as subscribed on both topics (so startup creates two rules, 1 one of each topic). Second case probably has less chance of unnecessary double processing of messages in our case.
I did notice that a migrated event (in code configuration) which still has its rule configured on the old event topic (since we donāt automatically clear these) will still pick up events from the old topic, effectively creating a doubly subscribed situation. While this could maybe work as a solution for us, it would feel a bit shaky that part of the subscription only works due to ālingeringā configuration, rather than being consistent on a fresh environment.
While it is possible to migrate events individually, it is currently not supported to partially migrate a delivery path of a single event.
This is based on the assumption that a mixed delivery path has all the downsides of the old topology and no benefits of the new one.
For autonomous team setup we recommend the following migration process
Each team upgrades to the latest version of the transport running the migration topology configured to common topic delivery (the āold topologyā).
Once all teams involved in a given event path (all publishers and subscribers) agree they want to migrate that path to the new topology, the migration process is kicked-off.
The teams that manage the subscribers need to act first, switching their configuration of the migration topology to topic-per-event. At this point the subscribers are receiving events thanks to the ālingering configurationā you mentioned but this should be fine because all the teams are committed to migrating sooner rather than later
The team that managers the publisher updates their configuration of the migration topology to topic-per-event. After this change is deployed, the events bypass the bundle-1 topic and the subscriptions there (the ālingering configurationā) can be removed
Would that process work for you? We believe the most important part is building the consensus about committing to the upgrade within some defined time window so that the system does not rely on ālingering configurationā forever.
I completely missed that function upgrade page when I was looking around, that was exactly what I was hoping for so I can add it as a link to our internal documentation.
The suggested migration process sounds like a solid plan. Iāll have to discuss a bit internally to see if we can build the required consensus to migrate specific paths within reasonable timelines, but I think this should probably work for us.