Azure-functions-durable-extension: Allow to get all available Durable Entities

[pacodelacruz]

Is your feature request related to a problem? Please describe.
I'd like to have a way to get the list of all existing Durable Entities. I can think of different scenarios where this could be useful. e.g.

• An application with multiple users who create entities, and the app administrator would like to know what entities have been created,
• The user not necessarily remembers the identifier of all entities that have been created.

Describe the solution you'd like
I'd like to have this available as part via the admin API and the client within the programming model. Additionally, some kind of paging might be useful.

Describe alternatives you've considered

• API: GET /runtime/webhooks/durabletask/entities/ Similar to https://github.com/Azure/azure-functions-durable-extension/issues/733 but instead of returning the state of a particular entity, return an array of identifiers of the available entities. Also, similar to https://docs.microsoft.com/en-gb/azure/azure-functions/durable/durable-functions-http-api#get-all-instances-status but instead of returning orchestrations, to return entities.
• Via the client, Similar to https://docs.microsoft.com/en-us/azure/azure-functions/durable/durable-functions-instance-management#query-all-instances, but instead of returning orchestrations, to return entities.

cc: @cgillum, as discussed previously via Twitter

[cgillum]

I agree we should do this. Here is a proposal I have put together for this, which I believe aligns with your idea (apologies for the internal implementation details mentioned here):

.NET query API

Let's assume we add an API named ListEntitiesAsync. The design of the API should be very similar to the design of the IDurableOrchestrationClient.GetStatusAsync(OrchestrationStatusQueryCondition, CancellationToken) API, which is our most flexible API for querying orchestrations. Internally, the ListEntitiesAsync API can still call the DurableClient.GetStatusAsync(...) API. However, it should modify the result that comes back so that it returns properties that are specific to entities.

Task<EntityQueryResult> IDurableEntityClient.ListEntitiesAsync(
    EntityQuery query,
    CancellationToken cancellationToken)`

```csharp
public class EntityQuery // similar to OrchestrationStatusQueryCondition
{
public string EntityName { get; set; }
public DateTime LastOperationFrom { get; set; } = DateTime.MinValue;
public DateTime LastOperationTo { get; set; } = DateTime.MaxValue;
public int PageSize { get; set; } = 100;
public string ContinuationToken { get; set; }
}

```csharp
public class EntityQueryResult
{
    public IReadOnlyCollection<EntityStatus> Entities { get; }
    public string ContinuationToken { get; }
}

```csharp
public class DurableEntityStatus
{
public EntityId EntityId { get; } // derived from InstanceId
public DateTime LastOperationTime { get; } // derived from LastUpdatedTime
public JToken State { get; } // derived from Input
}

One note about the `EntityQuery.EntityName` filter: DurableTask.AzureStorage doesn't have any way to filter by part of the instance ID name. We have two options for implementing this:

1. Don't try to filter by instance ID at all when calling storage. Instead, do in-memory filtering after the results come back. This can potentially be very inefficient, but it's very simple.
2. Update DurableTask.AzureStorage so that the `OrchestrationInstanceStatusQueryCondition` class and add an `InstanceIdPrefix` property. When it queries table storage, it can use this as a partition key filter. Details of how to use the Azure Storage API to do a prefix query are left as an exercise to the reader. :)

We should probably do option (2) since that will be the most scalable. If we do this, we can even consider making it an option for orchestration queries since some customers may benefit from this.

## HTTP API
The HTTP API should internally just call the .NET API. The design should match the [Get all instance status](https://docs.microsoft.com/en-us/azure/azure-functions/durable/durable-functions-http-api#get-all-instances-status) API, but modified to look like an entity API:

```http
GET /runtime/webhooks/durableTask/entities
    ?taskHub={taskHub}
    &connection={connectionName}
    &code={systemKey}
    &lastOperationTimeFrom={timestamp}
    &lastOperationTimeTo={timestamp}
    &fetchState=[true|false]
    &top={integer}

x-ms-continuation-token: XXX

[petr-stupka]

Hi, will be possible to implement Entity Class name filter?

The scenario is that i have multiple entity inventory classes and i would like to get all entities for specific class name only.

[cgillum]

@petr-x yes, entity class name filters will be supported. This feature will go out with the upcoming v2.1.0 release.