Azure-functions-durable-extension: [Feature Request] Orchestration Auto-Purge

[mpaul31]

** Let me know if this is not the right place to request a feature and I'll be happy to move.

It would be really nice to have a configuration setting in host.json to have orchestrations automatically clean-up themselves similar to how the PurgeInstanceHistoryAsync method works today.

I'm sure there are many production applications in-flight as we speak that overlooked this and would rather have this happen in the background.

Thanks and keep up the great work on this awesome product!

[cgillum]

Thanks for opening this issue! We'll look into this.

[b10-dslappendel]

Is there any idea when this will be worked on? We currently have issues where manually purging on a timer hits the function execution time limit continuously. Having it auto purged would be great for successful runs.

[cgillum]

I've added this issue to be scheduled for our next release. I think it makes a lot of sense for us to have this feature.

[olitomlinson]

In my use-case, I use a trigger on a 15 minute schedule to purge orchestrations once they have been in a completed runtime state for longer than 2 weeks. This allows me 2 weeks of diagnostic time to investigate any of my orchestrations in production.

However for orchestrations that are Running or Failed, I never delete these.

So I would suggest having configurable time periods for each runtime state, ranging from

• instantly purge the orchestration once it arrives at a particular state
• wait for x timespan to purge the orchestration once it arrives at a particular state.

—-

Please also emit a ‘customMetric’ which contains A count of how many orchestrations were purged, and their runtime state.

[sebastianburckhardt]

If we establish some process for periodic maintenance operations, It may make sense to also support automatic cleaning of entity storage (which currently requires an explicit API call, see #1442).

[amdeel]

Feature Proposal

Implement an Automatic Storage Cleanup feature that runs on a set time to perform cleanup maintenance operations for Durable Functions storage. It will call PurgeInstanceHistoryAsync and CleanEntityStorageAsync to delete Orchestration and Entity history, but could be expanded in the future for other cleanup operations.

We could make the settings configurable in the users host.json. I included some potential default settings:

bool UseAutomaicStorageCleanup -- (default true)
Timespan CleanupTimespan -- frequency to run cleanup operations (default 1 day)
Timespan PurgeOrchestrationsOfAge -- setting to purge orchestration instances of at least this age (default 2 weeks)
IEnumerable PurgeOrchestrationsWithStatuses -- (default Completed)

Considerations

Where do we implement this feature?

Durable extension

• One implementation regardless of storage backend.
• No guarantee that only one worker will call this cleanup operation.

DurableTask.AzureStorage

• Could leave open the possibility to expand on this in the future if lease management is necessary to ensure only one worker calls the cleanup operation.
• Every storage backend would need to implement this.

Edit: Some offline discussions lead us to deciding to implement this feature in DT.AzureStorage.

[anel-al]

@cgillum :)

It would be great to have this feature as part of platform.

While we are waiting, what is current official recommendation for application with large amount of history data? Timer based function with call to delete history API?

Thank you