Azure-functions-durable-extension: DurableOrchestrationClient.GetStatusAsync() to get count of instances per RunTimeStatus.

[thomas-schreiter]

I want to retrieve the count of all orchestration instances that are currently Completed / Pending / Running / etc. so that I can show it on my health dashboard. Currently, the only way to do this is to call DurableOrchestrationClient.GetStatusAsync() without parameters and then analyze the result, e.g. with a Linq query:

        var instances = await orchestrationClient.GetStatusAsync();
        var stats = instances.GroupBy(x => x.RuntimeStatus).Select(group => new 
        {
            Status = group.Key.ToString(),
            Count = group.Count(),
        }).ToList();

This takes up a lot of time and memory. In my case there are thousands of completed instances, so this take multiple minutes or doesn't even finish before Functions times out.

To speed things up, is there a way to run the grouping and counting server-side, so that the client doesn't have to pull the whole instance history?

[thomas-schreiter]

ADDENDUM. There is an unsupported way to retrieve the current status, which is loading the status from the DurableTaskHubInstances table. This seems to be a little bit faster. But I don't want to rely on it, because it's unsupported.

[cgillum]

[Thinking out loud] A server-side implementation would have the same potential issue with using a lot of memory.. Internally I think the way we would create a summary function like this would be to use paging and count instances as we go through the pages. In theory, if we supported paging, you could do the same thing on the client, though it might actually be more efficient to implement on the server.

FYI @TsuyoshiUshio this seems like it would be a generally useful feature if you are interested.

[cgillum]

One improvement that we've added is the ability to filter based on runtime status (amongst other things) via this GetStatus() overload. It won't allow you to do aggregation, but it may reduce the overall overhead of counting instances.

[thomas-schreiter]

That's great. I have been using that API and it works well for my purposes. Thanks.

[cgillum]

Awesome. Thanks for confirming. I'll go ahead and close this issue then.