Azure-functions-host: Setting maxConcurrentRequests to 1 does not work

[upamanyus]

@jaylorch and I have tried to run an application that is CPU-bound on Azure Functions, and found that setting maxConcurrentRequests to 1 does not limit the number of concurrently running requests to 1.

Tagging @pragnagopa to get the right folks for this issue.

• Timestamp: 2020-08-21T21:13:05.3540926Z
• Function App version: 3
• Invocation ID: dfc21e2b-f960-4a47-bb47-a41eca3dea93
• Region: East US

Repro steps

Use a hosts.json like

{
    "version": "2.0",
    "extensions": {
        "http": {
            "routePrefix": "api",
            "maxOutstandingRequests": 200,
            "maxConcurrentRequests": 1,
            "dynamicThrottlesEnabled": true
        }
    }
}

and use code like the example below. Then, make a big batch of concurrent requests (e.g. make 20 reqs in parallel).

Expected Behavior

Since the hosts.json has maxConcurrentRequests set to 1, Azure Functions should only process one request on an instance at a time.

Actual behavior

Azure Functions attempts to concurrently process requests on one instance, which causes our function to run out of memory and results in slow response times (because many invocations of our CPU bound program are being run concurrently). In our function, each request should use a few hundred MB of memory and take not more than 10-20 seconds to complete.

Known workarounds

We could try to detect when our function is being run concurrently, and force our code to wait for a currently-executing function to finish, but this would mean we get billed for the time spent waiting and might prevent Azure Functions' scaling logic from recognizing that we require more instances to process our requests.

Example code

Example code

```csharp

public static class ComputeBoundFunction
{
static bool AlreadyRunning = false;

[FunctionName("ComputeBoundFunction")]
public static IActionResult Run(
    [HttpTrigger(AuthorizationLevel.Anonymous, "post", Route = null)] HttpRequest req,
    ILogger log, ExecutionContext executionContext)
{
  if (AlreadyRunning) {
    var result = new ObjectResult("Already running on this instance!");
    result.StatusCode = StatusCodes.Status418ImATeapot; // HTTP 418 means concurrent request
    return result;
  }
  AlreadyRunning = true;

  // Run compute-bound process here;

  // Can simulate running time to see concurrent requests by un-commenting
  // Thread.Sleep(5000);

  AlreadyRunning = false;
  return new OkObjectResult("Successfully finished");
}

}

[benbelow]

I've also seen this, with EventGrid triggered functions.

In addition, the per-function scaling limit is also not working - leaving us with no way to limit concurrency, which is a major concern

[brettsam]

+@mathewc -- I can see in our logs that MaxConcurrentRequests is, indeed set to 1 for this app (ruling out any host.json parsing issues). Are there any issues with setting this to 1? DataFlow should handle that, right?

{
  "DynamicThrottlesEnabled": true,
  "MaxConcurrentRequests": 1,
  "MaxOutstandingRequests": 200,
  "RoutePrefix": "api"
}

[brettsam]

@benbelow -- which scaling limit are you referring to? This: https://docs.microsoft.com/en-us/azure/azure-functions/functions-scale#limit-scale-out?

If so, that would be a separate issue -- can you create a new issue in this repo with those details?

[msftbot[bot]]

This issue has been automatically marked as stale because it has been marked as requiring author feedback but has not had any activity for 4 days. It will be closed if no further activity occurs within 3 days of this comment.

[jaylorch]

It seems @benbelow is unable to respond. But the issue @benbelow raised is only tangential to the key reason for this issue: the fact that the parameter maxConcurrentRequests is being ignored. So let's remove the "Needs: Author Feedback", which was put in to ask @benbelow to provide feedback, to prevent the issue from getting deleted by msftbot.

[pragnagopa]

@jaylorch - thanks for flagging this. I removed Needs: Author Feedback

@brettsam / @mathewc - Any ideas on what to check to understand why maxConcurrentRequests is not being honored when host in Azure?

[itshemant09]

@pragnagopa @brettsam Even I observed the same. Any updates on this?

[brettsam]

The only thing I can think of is that DataFlow somehow isn't handling 1 correctly here, but I didn't see anything in the code that looked incorrect. We've gotten a few pings on this, so I'm going to move it to our next Sprint. @mathewc -- is this something you could take a look at?

[itshemant09]

Thanks @brettsam for taking this up. really appreciated

[mathewc]

I was able to repro this and am investigating a fix

[mathewc]

PR out for fix: https://github.com/Azure/azure-functions-host/pull/6867

[jaylorch]

Thanks for fixing this! When will the code with this fix be deployed to Azure Functions? I tested it just now, and it doesn't seem to be working on Azure Functions yet.

[pragnagopa]

@jaylorch - fix is checked in to Sprint 88 and will be part of next functions runtime release. ETA is end of December due to multiple deployment restrictions coming up in November and December.

[pragnagopa]

Fix is deployed to Azure and is part of runtime version https://github.com/Azure/azure-functions-host/releases/tag/v3.0.15193