when a set of messages(say 10) are hit the input queue at this point the sql server service is down, the messages are retried thrice and then entered into errorqueue in an order - Now will this order in which the messages have entered the errorqueue, will they be the same order in which the messages were processed in the input queue…? Because, the client, when the messages hits the error queue tries to drag them and drop into the input queue for reprocessing, by sorting them with msg#(by using QueueExplorer) and can i have figured it out, the orders are jumbled and 2 messages fails to process due to the order…
Hi Mary,
the order of messages is not guaranteed and depends on many factors e.g. concurrency limit, transport used, number of physical instances. It is only in very specific configurations when any assumptions about message ordering can be made. This includes not only failing messages but also those in the input queue that are getting processed just fine.
In the approach that you describe won’t probably work either. For example order in which messages are moved the the error queue is not guaranteed.
Can you tell me more about your endpoint setup and reasons why you need message ordering in the first place?
Regards,
Tomasz
We have 2 endpoints configured say A and B and below is my transport config:
Since, we have set NumberOfWorkerThreads=1 , I’m expecting the messages to be processed sequentially.
Server A is communicating the ORM entity updates to server B via messages to keep both servers in sync. The entities are saved to db on destination endpoint B, hence messages/entities need to be processed in correct order to avoid referential integrity errors.
When the SQL service at B is down, the messages hit the error queue. (Messages in error queue are not in expected order) When the SQL service is up and the client moves the messages from error queue back to input queue to re-process failed messages, the messages fail to process due to the order being incorrect.
Which they are, until messages are retried. We do not support ‘hold the line’ type of message processing.
However, you can build your own kind of feature if you wish. Have the sender add an counter to each messages send so that you can actually verify the correct order. On the receiving end buffer the received messages if required. The following (buffering) saga gives you an idea on how you can do this:
In general this works as messages are processed in order, thus the data does not need to be buffered.
In message based systems its best to assume on out of order delivery and rely on eventual consistency. Both of these should be dealt with on the query side. This way you don’t need referential integrity in your data store.
Another option is not so to send too small of messages. Send aggregates, sort a document, which contain a whole snapshot of a structure. Any referential integrity issues would be solved because the snapshot contains all relevant data.
It is not advisable to build your own replication using a messaging system. You should use low-level high performance replication features provided by your database vendor.
– Ramon