Postgres SQL Persistence - should events be publishable to two consumers which use the same DB?

, , ,
1

Hi,

If I’ve got two endpoints, which use the same persistence DB, both using PG transport/persistence (again, same DB), and where both appear against the same event in the subscription table (and subs look correct to me), am I correct in thinking that both endpoints should receive that event once published (by another physical service)?

Because that isn’t happening - it seems to only publish to one or the other. It’s not doing round-robin - it will literally only publish to one or the other, seemingly, whichever started first.

Just wanted to check - happy to raise an issue on GitHub if required. Thought it best to start here though.

In the meantime, I’ll have a play with my configurations to see if it’s something specific that’s causing this behaviour.

Thanks! Rob

2

I think I’ve figured out how to stop this behaviour. I had outbox enabled on all three endpoints. I disabled outbox and it now consistently publishes/handles both copies of the message.

I suspect it relates to my question here - PG Transport + PG Persistence + Transactional Session - where I wasn’t sure exactly how I should be setting this up. But for sure, disabling outbox has fixed this issue, and from what I can see, ITransactionalSession still works as it should on the publisher, from what I can tell.

What I suppose would be interesting to know is why enabling outbox caused this behaviour. What I did see was, when I turned off the consumers, the message was written to both queues, but then when I turned the consumers back on, only one of them received it. Almost as if some kind of idempotency check kicked in?

3

Just doing a bit more reading, wondering if it’s the de-dupe phase of Outbox that’s causing this? It shouldn’t do the because, as per the documentation, messages sent (published, in this case) from outside of a message handler (gRPC service using ITransactionalSession in my case) should have unique IDs? Feel like I need Outbox given I’m “below Transaction Scope” with PostgreSQL…

4

@robjohnson1978,

If I understand your scenario correctly, that sounds very off. They should both receive the message.

Can you check the outbox table names in the database? Each endpoint should have its own outbox table, named after the endpoint name.

5

@mauroservienti funnily enough, it just occurred to me that this is probably the issue. Currently there’s only a single outbox for all three endpoints. I’m literally in the middle of fixing that. Apologies in advance if that’s what it is…

6

Yep, @mauroservienti that was it. I had:

persistence.TablePrefix(string.Empty);

So… yeah. Feel like, given I’ve copied that from prod systems we have, I should probably raise a ticket on my side now.

One thing I would say though… I don’t see anything in the docs that explicitly states that “each endpoint needs its own outbox”. I suppose, logically, that’s inferred though. But I know for a fact that we share Outboxes amongst multiple endpoints, as I say, in our prod systems… :grimacing: