Azure-functions-host: CORS broken for Linux consumption plans

[jacobgunnarsson]

I'm running a node Azure Functions app on a Linux consumption plan and since sometime over the weekend preflight requests have stopped reaching the corresponding HTTP triggers.

Investigative information

• Function App version (1.0 or 2.0): 2.0
• Invocation ID: I can't provide invocation ID since I cannot see the request anywhere in the Azure Portal
• Region: West Europe

Repro steps

Make a preflight request to any HTTP trigger, presumably on a Linux consumption plan. As an example:

curl -i '<any-azure-function-http-trigger-url>' -X OPTIONS -H 'Connection: keep-alive' -H 'Pragma: no-cache' -H 'Cache-Control: no-cache'  -H 'Origin: <any-origin>' -H 'Access-Control-Request-Method: GET' -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.70 Safari/537.36' -H 'Access-Control-Request-Headers: authorization' -H 'Accept: */*' -H 'Sec-Fetch-Site: cross-site' -H 'Sec-Fetch-Mode: cors' -H 'Referer: <any-referrer>' -H 'Accept-Encoding: gzip, deflate, br' -H 'Accept-Language: en-US,en;q=0.9,sv;q=0.8,la;q=0.7' --compressed

Expected behavior

I expect the request to reach my HTTP trigger where I can process it.

Actual behavior

The request never reaches the HTTP trigger. I can't see the request in any Azure Portal monitoring tools.

I get a response that looks like this:

HTTP/1.1 204 No Content
Server: Kestrel
Request-Context: appId=<app-id>
Date: Mon, 04 Nov 2019 10:46:27 GMT

Known workarounds

No known workarounds. Tried using proxies.json to override headers, but the preflight request never reaches the proxies.

Related information

Removing any relevant preflight request header, such as Access-Control-Request-Method: GET makes the request successfully reach the corresponding HTTP trigger.

[jacobgunnarsson]

Please let me know if you need any more information and I'd be happy to provide it.

[jacobgunnarsson]

Seems like the preflight request is being caught by some middleware. Maybe related to this release?

Poking around some more it seems like I can add allowed origins by using the CLI (az functionapp cors) which in turn makes the preflight response include the proper Access-Control-Allow-Origin header; like so:

Access-Control-Allow-Origin: <allowed-origin>
Date: Tue, 05 Nov 2019 13:21:09 GMT
Request-Context: appId=<app-id>
Server: Kestrel
Vary: Origin

The preflight request still fails, because I cannot configure any other required preflight response headers such as Access-Control-Allow-Headers or Access-Control-Allow-Methods and so on.

I know the documentation says to configure CORS in the Azure Portal, but running on a Linux consumption plan the CORS option is greyed out under Platform features, saying _This feature is not supported for Linux apps on a Consumption plan_

This is really confusing.

Edit: To clarify, before this problem arose, I simply handled preflight requests in the HTTP triggers themselves. This worked just fine, but is impossible now since the request gets swallowed sometime before the trigger, effectively breaking CORS entirely.

[ericdrobinson]

We appear to be experiencing this as well. The last reports we have of the function (Python on Linux Consumption) working as expected was Friday, November 1st. All calls to the HTTP function from _browsers_ result in "No Content" (I emphasize browsers because a simple curl request for OPTIONS actually returns the expected headers).

Edit: To clarify, before this problem arose, I simply handled preflight requests in the HTTP triggers themselves. This worked just fine, but is impossible now since the request gets swallowed sometime before the trigger, effectively breaking CORS entirely.

Exactly the same for us. This is how we handled it in our Python HTTP trigger:

if req.method.lower() == "options":
    logging.info("Handling OPTIONS HTTP verb request.")
    return func.HttpResponse(headers={"Access-Control-Allow-Origin": "*", "Access-Control-Allow-Methods": "GET, POST", "Access-Control-Allow-Headers": "Content-Type"})

[ericdrobinson]

@jacobgunnarsson I did a little bit of testing and discovered a minimum repro for your curl example.

Specifically, if a preflight request includes the following two headers, the request never makes it to the function for processing:

1. Access-Control-Request-Method
2. Origin

This means the following command will reproduce the issue:

curl -i -X OPTIONS <azure-function-http-trigger-url> -H "Access-Control-Request-Method: GET" -H "Origin: https://www.example.com"

but that neither of the following will (they will fall back to your function's custom preflight code):

curl -i -X OPTIONS <azure-function-http-trigger-url> -H "Access-Control-Request-Method: GET"
curl -i -X OPTIONS <azure-function-http-trigger-url> -H "Origin: https://www.example.com"

Can someone _please_ address this issue quickly? So far as we're aware, there is nothing that we can do on our end to address or otherwise work around this issue.

[balag0]

@jacobgunnarsson Yes. the latest release adds support for CORS. Portal update to enable the cors feature is scheduled for next week. Like you mentioned 'az functionapp cors' can be used to configure CORS until then.
Are you still seeing the issue after configuring the CORS to include the additional origins for the app?

@ericdrobinson Are you seeing the issue only for OPTIONS requests? Are you using CORS in your function app?

[jacobgunnarsson]

@ericdrobinson thanks for looking into this in a bit more detail!

@balag0 thanks for your reply! Yes, the issue still persists. As I wrote before adding allowed origins through Azure CLI (or through the portal) properly adds the Access-Control-Allow-Origin header to the preflight response.

However, the issue is not wether or not the Access-Control-Allow-Origin header is present in the preflight response. The issue is that the rest of the required headers are missing, what about Access-Control-Request-Method or Access-Control-Allow-Headers? (or any of the other Access-Control-* headers, described here)

If the preflight request includes those headers but the response does not, the response is considered unsafe and is rejected by the browser.

[ericdrobinson]

@balag0 Some responses:

the latest release adds support for CORS.

It seems that it doesn't just "add" support, but enforces it. And it does so in a broken manner (it is incomplete insofar as required header reporting is concerned).

Portal update to enable the cors feature is scheduled for next week.

Why in all the heavens would you guys push something live that is both enabled by default and inserts itself in the existing flows without _at least_ sending out a warning to users first? (Especially when there is only one expected approach to work for the time being...)

Like you mentioned 'az functionapp cors' can be used to configure CORS until then.

But _also_ as was mentioned by @jacobgunnarsson:

The preflight request still fails, because I cannot configure any other required preflight response headers such as Access-Control-Allow-Headers or Access-Control-Allow-Methods and so on.

The az functionapp cors command doesn't work for CORS preflight. This means that you guys have inserted some code in a location that we cannot modify or workaround. To be clear, the workaround we implemented (the code in my previous comment) was modified from the code suggested in the official functions-reference-python documentation.

Given the default CORS handling implementation in both the python functions documentation and the Azure CLI cors add call, it appears that the system is set up to expect only Simple Requests. It does not appear to be set up to properly handle Preflighted Requests.

What's the difference that causes browsers to use the Preflighted approach instead of the Simple one? Payloads. Attaching a body to a message to a request to a different origin than the initiating resource (HTML/JavaScript) will cause the browser to check with the server if the server is cool with accepting requests of the given format. In our case, this requires that the following headers are set:

1. Access-Control-Allow-Origin
2. Access-Control-Allow-Methods
3. Access-Control-Allow-Headers

All _three_ of those must be properly configured for the Preflight check to pass muster. It looks as though the current CORS implementation isn't built to allow this.

Is there something that we're missing? Is there a way to turn disable automatic CORS processing until the underlying system can be properly configured to handle such preflight requests?

[ericdrobinson]

@balag0 With respect to your questions:

Are you seeing the issue only for OPTIONS requests?

Umm... I'm honestly not sure how to respond to this. The _browsers_ (tested: Chrome, Safari, Firefox) all _fail_ to send anything beyond the Preflight OPTIONS request. In actuality, they REFUSE to send anything beyond the initial OPTIONS because the response they receive from Azure effectively says "not supported":

Access to XMLHttpRequest at '[app url]' from origin '[origin domain]' has been blocked by CORS policy: Response to preflight request doesn't pass access control check: No 'Access-Control-Allow-Origin' header is present on the requested resource.

To be clear, we have not configured CORS via the Azure CLI yet. Given @jacobgunnarsson's reports and the way that CORS Preflight responses are expected, we have zero expectations that our experience will differ at all from @jacobgunnarsson's with respect to the preflight requests being "okayed". Regardless, I will give it a shot and report on my findings.

Are you using CORS in your function app?

What do you mean by this question?

[ericdrobinson]

Regardless, I will give it a shot and report on my findings.

Yup. I just ran the test and the browser refuses to send anything beyond options. The error messages now read:

Access to XMLHttpRequest at '[app url]' from origin '[origin domain]' has been blocked by CORS policy: Request header field content-type is not allowed by Access-Control-Allow-Headers in preflight response.

I'd be willing to bet that if you added support for Access-Control-Allow-Headers that the next error to appear would be that Access-Control-Allow-Method didn't specify POST (or whatever method was used in the request).

[balag0]

thanks for all the details. investigating this more today and will update here soon.

[ericdrobinson]

A simple way to get a browser to trigger the Preflight Request is to issue a request (XHR; Fetch) that includes a payload (body). This example (adapted from MDN's Fetch API documentation) should do the trick:

const url = 'https://[someappname].azurewebsites.net/httptrigger';
const data = { username: 'example' };

try {
  const response = await fetch(url, {
    method: 'POST', // or 'PUT'
    body: JSON.stringify(data), // data can be `string` or {object}!
    headers: {
      'Content-Type': 'application/json'
    }
  });
  const json = await response.json();
  console.log('Success:', JSON.stringify(json));
} catch (error) {
  console.error('Error:', error);
}

Hopefully this is helpful.

[balag0]

Definitely a bug in how we configured the CORS middleware. Opened a PR https://github.com/Azure/azure-functions-host/pull/5215 to fix this.

We should be able get this deployed sometime early next week.

@ericdrobinson @jacobgunnarsson Thanks for the patience and all the help investigating this.

[swantzter]

I have an issue related to this as well, I think. I have my function set up to handle CORS for me (stripped down function config below)

However since Nov 1 or thereabout the functions host doesn't let OPTIONS requests through to my function and instead just responds to OPTIONS requests with this:

HTTP/1.1 204 No Content
Server: Kestrel
Request-Context: appId=cid-v1:<UUID>
Date: Fri, 08 Nov 2019 15:29:10 GMT

It feels like this is related to the addition of this native CORS support?

function.json:

{
  "bindings": [{
    "authLevel": "anonymous",
    "type": "httpTrigger",
    "direction": "in",
    "name": "req",
    "methods": ["options"]
  }]
}

[ericdrobinson]

@svbeon Yup. Looks like you're affected by this issue. According to @balag0, this has been fixed and will probably be live early next week.

[balag0]

Update: The build with the fix is available now, the plan is to include this change with the next scheduled release starting end of the week instead of doing an one off deployment today.

[ericdrobinson]

@balag0 Our functions still aren't working properly. We're about two weeks with broken features now. Is there a useful ETA for a fix deployment that you can share?

[balag0]

Apologize for the delay. I will update here as soon as i have more information.

[swantzter]

@balag0 Any news, we've had to delay launch for a couple of weeks now and it's becoming costly... should we start looking at alternatives to the linux-specific code we have been using or will this get fixed this week?

[balag0]

Fix is deployed on all regions. Please give it a try (might need an app restart if the app has been running non-stop).

[swantzter]

@balag0 thanks for the update. it seems like I might be having some other issue then.
It is letting OPTIONS requests through, yntil i set an Origin header, then I get "empty headers"

Server: Kestrel
Request-Context: appId=cid-v1:xxxx
Date: Tue, 26 Nov 2019 09:55:42 GMT

Do I need to set anything but leaving all fields blank for CORS in the azure portal for it to work?

[balag0]

@svbeon Yes sounds like a different issue. Do you mind opening a new issue. Thanks.

[ericdrobinson]

Fix is deployed on all regions. Please give it a try (might need an app restart if the app has been running non-stop).

Our functions are working again. Not sure if this is falling back to our own CORS implementation or being handled properly by the host (likely the host, at this point). One way or the other, browsers are no longer dropping the responses due to incorrect CORS responses.

[irecevic]

@balag0 thanks for the update. it seems like I might be having some other issue then.
It is letting OPTIONS requests through, yntil i set an Origin header, then I get "empty headers"

Server: Kestrel
Request-Context: appId=cid-v1:xxxx
Date: Tue, 26 Nov 2019 09:55:42 GMT

Do I need to set anything but leaving all fields blank for CORS in the azure portal for it to work?

I can confirm the same behaviour. The Chrome is setting in preflight origin and expects 'Access-Control-Allow-Origin' response.

The error is:
Access to XMLHttpRequest at 'https://xxxxxx.azurewebsites.net/api/HttpTrigger' from origin 'https://www.default.com' has been blocked by CORS policy: Response to preflight request doesn't pass access control check: No 'Access-Control-Allow-Origin' header is present on the requested resource.

The workaround is to add via CORS (using shell) allowed origin

az functionapp cors add --allowed-origins https://www.default.com --resource-group xxxx --name xxxxx

[balag0]

Please open a new issue and post this information there for easier tracking. closing this issue since this is for a different issue.