Azure-functions-durable-extension: APIs to enumerate all orchestrations

[TsuyoshiUshio]

Hi @cgillum ,

As we talked, customer needs on the fly orchestrator status as simple as possible.

We can develop it now since we've got DurableFunctionsHubInstances, we can implement it as a method of Orchestrator client or REST API. I'd like to contribute this feature since my customer wants it.

[k-miyake]

If we can get whether running status exists, Durable Functions's DevOps control will be simplified. For example, a situation of decide whether to start swap slots.

[cgillum]

Excellent, I would love to receive help with this. The work will need to be done in multiple parts, one part in the Durable Task Framework and one part in the Durable Functions extension. Here is my design suggestion:

DurableTask.AzureStorage changes:

Lets update ITrackingStore with a new GetStateAsync overload, like this:

Task<IList<OrchestrationState>> GetStateAsync();

This interface is implemented by two concrete classes, AzureTableTrackingStore and InstanceStoreBackedTrackingStore. You only need to add logic for AzureTableTrackingStore. The other classes implementation can throw NotImplementedException. As an optimization, maybe we can add some filtering parameters, but it's probably not required in the first version.

This new GetStateAsync overload is different from the others because it will query all orchestration instances in the task hub, not just one instance. It will need to scan the Instances table using the Azure Table Storage API to return all rows and convert each row into an OrchestrationState object. You can use the existing GetStateAsync implementation as a reference (maybe both methods can use the same common code).

Next, we'll need to update the AzureStorageOrchestrationService class with a new public method, like Task<IList<OrchestrationState>> GetStateAsync(). This will be called by the Durable Functions extension to get the list of instances. This is not the best design, because it is specific to Azure Storage and cannot be used by other providers (like DurableTask.ServiceBus, but we can fix that later. Internally this new public method will call your new GetStateAsync method.

Let's make sure to create tests for this new logic as well in the DurableTask.AzureStorage.Tests project.

Microsoft.Azure.WebJobs.Extensions.DurableTask changes:

TODO: I will add more information about how to create the new REST API here later.

Let me know if you have any questions or get stuck. @gled4er can also help since he is familiar with how to change this code.

[TsuyoshiUshio]

Hi @cgillum
Thank you for your suggestion! I'll start to open a pull request and implement that. :)

[TsuyoshiUshio]

Hi @cgillum
One question. At the DurableTask repo, which branch should I folk from? There is no dev branch and I can see a lot of branches.

[cgillum]

Ah, that's a very good question. The Durable Task Framework repo is not as well organized. I have just now created a branch named azure-storage. Please create your fork from there.

[TsuyoshiUshio]

@cgillum Thanks! :)

[cgillum]

PRs are merged and I've signed off on the documentation.