In our system we receive xml and send xml files from and to external clients.
Most of the xml files (say 90%) are small, less than 25kb. We also receive messages with larger content, and I estimate about 1% that are a couple of MB’s (say 2-8 MB).
Every now and then (literally about 4/5 times a year) we receive files that are 10+ MB. The maximum size is 25MB. In our current set up, the xml content is sent in events as payload using a the NServiceBus Data Bus. We try to break down the xml in smaller bits that we can process seperately, but that is not always an option.
In the past the system used MSMQ as it’s transport, and with the message limit of 4MB we had to use the Data Bus. But recently we have switched to RabbitMQ which would allow larger message sizes, but the setup is still with the Data Bus.
Using the DataBus comes with advantages (smaller messages) but also disadvantages (more complexity, clean up of the data bus files).
It seems that we could now completely remove the Data Bus and hold the content of the xml files directly in the events. I am wondering if there are any reasons to stick with the Data Bus or things that I should consider.
- Is it better to stop using the Data Bus and keep the data on events? (and accept that a few messages will be large)
- Even though there will not be many large events, what can I expect in terms of performance in general? Will it take a hit (more memory usage for instance) or a boost (no need to retrieve files from the Data Bus)?
- How will this affect monitoring/audit? Anything to consider from that perspective?
- Other things I should take into account?