Orleans: Recommended way to Handle Collections of Grains

[RehanSaeed]

I'm new to Orleans and am trying to work out how to deal with collections of grains e.g. A UserGrain may contain many BlogPostGrain's and this requires filtering, sorting and paging etc. What is the recommended approach?

For small collections, I suppose I could just use a List<T> but what about for larger collections? Also, how big does the collection need to get before I need to think about stopping the use of List<T>? If I'm using Azure Table Storage, I have a maximum state size constraint which is what I'm guessing is my limit in this scenario.

I've seen two projects that seem to have some promise but they are supposedly not ready for production:

• https://github.com/OrleansContrib/OrleansV2.Fork.Indexing (pre-production = beta?)
• https://github.com/OrleansContrib/Orleans.Containers (pre-alpha)

[JorgeCandeias]

Until the Indexing project is live, it looks like the most practical option is still to issue direct calls against the storage system. One can also implement a registry pattern to make it look prettier and help keep the data in memory, if the use case allows.

[SebastianStehle]

Even with the index system I would argue that direct calls to the storage system are the most efficient solution, especially if you only want to read data.

I see it as a CQRS solution, where Orleans covers the write side.

[RehanSaeed]

How far off is the indexing project? Is it alpha or beta quality? Can we use it now?

So far, I have only seen mention of CosmosDB which indexes all properties. Does it also have support for Azure Table Storage? One of Orleans main selling points for me is that it's so cheap to run when combined with Azure Table Storage. If so, how would this even be implemented? The only way I can think of is to have multiple copies of the data for each property you want to index by.

Interesting comments about going direct to Storage. How would you do that with Azure Table Storage?

[RehanSaeed]

Some of my questions have been answered in https://github.com/dotnet/orleans/issues/602.

[philbe]

@SebastianStehle: The CQRS pattern will work fine. It will be easiest to implement when storage has built-in indexing (e.g., Cosmos DB rather than Azure Table). Still, some care is needed, since a query will return a snapshot that isn't updated when the underlying grain state changes. By contrast, the indexing project ensures that an index grain (in memory) is kept consistent with the underlying grain state. That is, the index is a materialized view, not a snapshot.

[sergeybykov]

@RehanSaeed Can we close this now?

[RehanSaeed]

I don't feel that Orleans has an answer to this question until the work in https://github.com/OrleansContrib/OrleansV2.Fork.Indexing is completed. Since that work is already tracked in https://github.com/dotnet/orleans/issues/602, feel free to close this.