Orleans: Streams for communication between different Orleans deployments

[DixonDs]

Will it possibly work, if I try to use a persistent stream provider in different Orleans deployments to pass messages from one deployment to another one? By different deployments I mean totally different codebase and servers, but obviously the message contract should be shared.

[galvesribeiro]

@DixonDs AFAIK you can do that. Just register the publishers in one cluster/deployment and the subscribers in another. In fact, I'll need it on my current project so I would be facing the same problem soon. Please let me know if you face any problems.

[DixonDs]

@galvesribeiro But I guess that will work only if you have subscribers in single Orleans deployment, right? I think if you have subscribers to the same stream in more than one Orleans deployment, the message could be removed by one Orleans deployment before the other one gets hold of it.

[galvesribeiro]

Yes, but I guess that depends on the provider's backend implementation... If you use Azure Queue Storage, for sure only 1 consumer will receive the message, since you need to mark it as readed and AQS remove the message from the queue. However, there are other pub/sub mechanisms like Service Bus which can have multiple consumers. I wonder if Orleans' implementation of Stream provider deal with that. I need to test.

In all cases, @jason-bragg is the Stream guru and he could put more input here.

[sergeybykov]

If you use a rewindable stream backed by log-based queues, such as EventHub or Kafka, multiple deployments can consume from the same queues independently.

Won't work with queueing systems, such as Azure Queues, that delete consumed events.