Azure-functions-durable-extension: Question: Orchestration Instance & Concurrency

[mpaul31]

If a orchestration (or sub-orchestration) is awaiting an activity does that orchestration instance count against the maxConcurrentOrchestratorFunctions? Or just when the orchestration is actually executing?

We have an orchestration that invokes an activity with a large exponential back off retry policy and just wanting some clarification in case we need to tinker with the maxConcurrentOrchestratorFunctions config value.

Thanks!

[cgillum]

An awaiting orchestration does not count against the maxConcurrentOrchestratorFunctions quota. That quota is really about how many orchestrations are actively being held in memory.

There is a small caveat to this - when you set extendedSessions to true in host.json, an awaiting orchestration will continue to remain in memory for up to 30 seconds after it has awaited. While it is in memory, it will continue to count against maxConcurrentOrchestratorFunctions. If no new messages arrive during those 30 seconds, the orchestration will be unloaded from memory and will no longer count against this quota.

[mpaul31]

Ok, so I get when extendedSessions is set to true, but assuming extendedSessions is set to false, there really is no concurrency limit since storage is being called for the orchestration state? Or is it back to what I thought only when it is only executing?

[mpaul31]

Hi @cgillum

Is my thinking about in-line with the intended behavior?

[cgillum]

I'm not sure what you mean when you say "there is no concurrency since storage is being called for orchestration state". There are concurrency limits for orchestrations. If you schedule 1,000 orchestrations to run at the same time and have maxConcurrentOrchestratorFunctions set to 100, then only 100 orchestrations will be allowed to use CPU cycles while the other 900 will be prevented from starting until currently running orchestrations either complete or await.

[mpaul31]

gotcha that makes total sense. when i said "no concurrency limit" i was thinking if we were not using extended sessions the state would not be stored "in-memory", thus the state would need to be retrieved from the storage account.

[mpaul31]

Sorry @cgillum one more question regarding this. Just wanted to make sure the following from the documentation is still true and has not possibly changed.

As an example, if durableTask/extendedSessionIdleTimeoutInSeconds is set to 30 seconds, then a short-lived orchestrator function episode that executes in less than 1 second will still occupy memory for 30 seconds. It will also count against the durableTask/maxConcurrentOrchestratorFunctions quota mentioned previously, potentially preventing other orchestrator functions from running.

This statement leads me to believe even when the orchestration completes the orchestration will remain in memory until the timeout period has elapsed.

[cgillum]

Completed orchestrations will always be removed from memory immediately. What I meant by "short-lived orchestrator function _episode_" is a single replay that doesn't result in the orchestration completing. Sorry if that was confusing.