Orleans: IStreamProviderFactory would be nice

[JanEggers]

hi there,

it would be nice if there was a IStreamProviderFactory like IGrainFactory so you could write extensions against it that work on the Client and in the Grain.

public interface IStreamProviderFactory 
{
        IStreamProvider GetStreamProvider(string name);
}

public interface IClusterClient : IStreamProviderFactory 
{
     ...
}

public abstract class Grain : IAddressable, ILifecycleParticipant<IGrainLifecycle>, IStreamProviderFactory 
    {
     ...
    }

[yevhen]

Another interesting api option is to introduce StreamReference similar to GrainReference which encapsulates stream provider name and stream id and get rid of IStreamProviderFactory->IStreamProvider->GetStream ceremony. That will also make mocking more streamlined.

[galvesribeiro]

Both very interesting suggestions... Indeed there is a big boilerplate at OnActivateAsync() in order to use streams...

[xiazen]

I want to understand where the interface design came from. What problem it tries to solve? Does it come from the pain points of using streams? What are those pain points? So I basically want to understand the problem, then visit the solution :)

I think you mentioned one pain point of "get rid of IStreamProviderFactory->IStreamProvider->GetStream ceremony". Is this method basically too long and annoy, or there's other paint points attached to it ?

[xiazen]

Is it for making test mocking easier? Or does it solve more than that?

[galvesribeiro]

get rid of IStreamProviderFactory->IStreamProvider->GetStream ceremony

and

making test mocking easier

Both are big annoying problems for the development experience.

In other words, make it easier to mock/test and remove the boilerplate to get a stream reference.

[xiazen]

I'm sorry , I'm slow here. I totally get the test mocking part.

but with IStreamProviderFactory, you still need to do grain.GetStreamProvider(xxx).GetStream dance. The call pattern is similar to what we have now. Are you suggesting with IStreamProviderFactory, you can build cleaner extension methods to make that dance shorter?
So basicly, I'm not entirely sure what do you mean by boilerplate to get a stream reference.

[galvesribeiro]

@xiazen the point is, why does the grain implementation need to know what provider it needs to use?

The grain code should be aware only about the stream id. The GetStreamProvider() call is totally not idiomatic. It could be registered at startup type a factory which gets injected to the grain and allow it to just providerFactory.GetStream(streamId).

Or even better. Don't need a factory to be injected at all. The stream reference could be directly injected as facet.

The way @yevhen suggested, to be similar to GrainReference, will bring us more one dependency on the inheritance of Grain, which is what we are trying to avoid (so in a dream day, we would have POCO grains!).

I think some sort of streamProvider -> grainType mapper would be registered at startup and generate a IStreamReference that would be injected to the grain when it is getting constructed.

[JanEggers]

I created the issue initially because I wanted to write one extension method that takes all required parameters to get a stream, that should work for clients and grains. but currently this is not possible because Grain and IClusterClient dont share a common interface for that. but in the end all i wanted is less code so what @yevhen suggested is a much better.

[galvesribeiro]

Less code is always good. :) However, we have to think on the consequences of having that code reduction...

[yevhen]

@xiazen I view streams much the same as grains (in fact each stream is backed by grain), Both are addressable entities. For grain it's a combination of type+id, for streams - provider+id. IStreamProvider has single method, and to get stream provider - it's another method. Combining both we can get rid of Grain->GetStreamProvider(name)->GetStream(id) dance and just have single function like Grain->GetStream(provider, id). IStreamProvider is superfluous and doesn't add any value for consumers.

[yevhen]

Moreover, if we go with StreamReference which encapsulates provider+id we can have an entity that could be passed around in the same way as you could pass GrainReference. Then, as all streams have uniform interface (Push/Subscribe) we can further combine IAsyncStream interface into StreamReference and have single entity, reducing cognitive burden. I think less types is better.

@xiazen Please, take a look at StreamRef in Orleankka. It combines properties of a reference and communication interface in a single entity that could be passed around. It's very easy to use and unit test (mock). Good example of usage could be seen here

[xiazen]

@yevhen thanks for the suggestion. We started to realize that stream has a lot similarity with grains. And with the streaming 2.0 discussion boiling up, it is possible that we move in that direction more in the future. Thanks for linking me those code. Definitely worth a look on how Orleanskka handles streaming. Let me take a look and get back to you and maybe trigger more interesting discussion. But that probably will be after Orleans 2.0, since that is the priority now.

But if we do move to something similar to StreamReference, it requires big changes to streaming. That needs more discussion and decision making than just IStreamProviderFactory interface. I think for @JanEggers 's concern, IStreamProviderFactory can fit in current framework without big changes. And it is a good idea.

@JanEggers I think the IStreamProviderFactory interface idea is nice. If you have time , feel free to submit a pr for it. if not, I can bring that up to the team. but it probably will be after Orleans 2.0, since it is the priority now.