Nodejs-storage: generateV4UploadSignedUrl CORS issue

[AnuroopSuresh]

Issue Browser from origin 'http://localhost:8100' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource both in firefox and chrome

My Node JS Code:

const admin = require('firebase-admin');
admin.initializeApp({
    credential: admin.credential.cert(serviceAccount),
    databaseURL: "https://***.firebaseio.com"
});
const bucket = admin.storage().bucket('gs://****.appspot.com');

async function generateV4UploadSignedUrl(filename, mimeType, expiryInMs) {
    // These options will allow temporary uploading of the file with outgoing
    // Content-Type: application/octet-stream header.
    const options = {
        version: 'v4',
        action: 'write',
        //expires: Date.now() + 1000 * 60 * expiryInMins, // 15 minutes
        expires: expiryInMs,
        contentType: mimeType,
    };

    // Get a v4 signed URL for uploading file
    const [url] = await bucket
        .file(filename)
        .getSignedUrl(options);

    console.log(`curl -X PUT -H 'Content-Type: ${mimeType}' --upload-file ${filename} '${url}'`);
    return url;
}

My Angular Code:

import {HttpClient, HttpHeaders} from '@angular/common/http';
import {switchMap} from 'rxjs/operators';

constructor(private httpClient: HttpClient) {

 }
this.httpClient.post<any>(url, postBody, {
            headers: {
                'Authorization': this.authToken,

            }
        }).pipe(switchMap(res => {
            console.log('home.page.res: ', res);
            return this.httpClient.put(res.url, this.fileToUpload, {
                headers: {
                    'content-type': 'image/jpeg',
                }
            });
        })).subscribe(res => {
            console.log('home.page.: ', res);
        });

My gsutil cors get gs://*.appspot.com

[{"maxAgeSeconds": 3600, "method": ["*"], "origin": ["*"], "responseHeader": ["*"]}]

Browser response

alt-svc: quic=":443"; ma=2592000; v="46,43",h3-Q050=":443"; ma=2592000,h3-Q049=":443"; ma=2592000,h3-Q048=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,h3-T050=":443"; ma=2592000
cache-control: private, max-age=0
content-length: 267
content-type: application/xml; charset=UTF-8
date: Fri, 03 Apr 2020 08:23:57 GMT
expires: Fri, 03 Apr 2020 08:23:57 GMT
server: UploadServer
status: 400
x-guploader-uploadid: AEnB2UqUuAZ-giq_ljMBwLrXNN5SRZHKwb12jI-uhhXU4oXczdEzjnZLjyOxXS3pQxYb3dUWG508_jui--wrM4xAnjQhGT33Mg

Issue:
No CORS headers like Access-control-allow-origin present.

Any assistance is much obliged.

Environment details

• OS: Windows 10
• Node.js version: 12.6.1
• npm version: 6.13.4
• firebase-admin: 8.10.0
• Google Chrome: 80.0.3987.149 (Official Build) (64-bit)
• firefox : 74.0 (64-bit)
• @google-cloud/storage version: using firebase sdk

Steps to reproduce

1. create a upload url from nodeJS server, send it as response
2. make a put request to the generated signed url in angular using httpClient

[jkwlui]

Hi @AnuroopSuresh, would you be able to try with "responseHeader": ["Content-Type"] in your CORS config?

[AnuroopSuresh]

@jkwlui Hello, sorry for the late reply, your suggestion is not working.
Browser Error message.
Access to XMLHttpRequest at 'https://storage.googleapis.com/...' from origin 'http://localhost:8100' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource.

My CORS Configuration, gsutil cors get output
[{"maxAgeSeconds": 3600, "method": ["*"], "origin": ["*"], "responseHeader": ["Content-Type"]}]

Browser header response from signed url
alt-svc: quic=":443"; ma=2592000; v="46,43",h3-Q050=":443"; ma=2592000,h3-Q049=":443"; ma=2592000,h3-Q048=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,h3-T050=":443"; ma=2592000 content-length: 825 content-type: application/xml; charset=UTF-8 date: Tue, 21 Apr 2020 07:08:00 GMT server: UploadServer status: 403 x-guploader-uploadid: AAAN...

Not working.

[jkwlui]

Hi @AnuroopSuresh, one possibility is that the CORS response might have been cached by the browser (for 3600 seconds). Can you try clearing cache on your browser and let us know if things changed?

Or alternatively, try with curl:

curl -v -i -X OPTIONS -H "Origin: hello.com" -H "Access-Control-Request-Method: PUT" -H "Access-Control-Request-Headers: Content-Type" <YOUR_SIGNED_URL>

Again, thanks for your continued patience while we figure this one out.

[AnuroopSuresh]

@jkwlui Hi, done complete "delete everything" clear in chrome and firefox, same error, funny thing. it suddenly worked once, but i try to make put request again, it failed. i cleared everything and retried. it failed. CURL is not helpful for me, i need it work for browsers

[febero]

Hi @AnuroopSuresh,

I ran into the same problem when trying to upload a file using a signed url via xhr. I solved this by adding X-Requested-With to the responseHeader in my Cloud Storage bucket CORS config.

Without it, the UploadServer returns a 200 OK response to the preflight request, but all of the Access-Control-[X] headers are missing.

My responseHeader config looks like this :

"responseHeader": [
      "Content-Type",
      "Access-Control-Allow-Origin",
      "X-Requested-With"
    ]

And for clarity's sake, this is my whole CORS config (make sure to replace the origin value) :

[
  {
    "origin": ["<yourOrigin>"],
    "method": ["OPTIONS", "PUT"],
    "responseHeader": [
      "Content-Type",
      "Access-Control-Allow-Origin",
      "X-Requested-With"
    ],
    "maxAgeSeconds": 3600
  }
]

Note that if OPTIONS is missing in the method array, the preflight request goes through nonetheless. @jkwlui I might be missing something here, but is it the expected behavior ?

[jkwlui]

You shouldn't need to include OPTIONS in the config, as it is implied implicitly. If you wouldn't mind confirming this: did your xhr request succeed with the X-Requested-With responseHeader field and failed if it wasn't present?

[jkwlui]

Thank you for the update @AnuroopSuresh.

The curl command is just to see if, making a preflight request (hence the OPTIONS method) would work outside of the browser setting, which might have the preflight request cached, so we can eliminate that possibility.

[AnuroopSuresh]

@febero Hi i modified the CORS Config, it worked for the first time, second time gave the same error. strange issue.

[AnuroopSuresh]

@jkwlui this is my CURL resposne, it seems i'm getting proper header in CURL
```
Trying 216.58.197.80...

• TCP_NODELAY set
• Connected to storage.googleapis.com (216.58.197.80) port 443 (#0)
• schannel: SSL/TLS connection with storage.googleapis.com port 443 (step 1/3)
• schannel: checking server certificate revocation
• schannel: sending initial handshake data: sending 187 bytes...
• schannel: sent initial handshake data: sent 187 bytes
• schannel: SSL/TLS connection with storage.googleapis.com port 443 (step 2/3)
• schannel: failed to receive handshake, need more data
• schannel: SSL/TLS connection with storage.googleapis.com port 443 (step 2/3)
• schannel: encrypted data got 2980
• schannel: encrypted data buffer: offset 2980 length 4096
• schannel: sending next handshake data: sending 93 bytes...
• schannel: SSL/TLS connection with storage.googleapis.com port 443 (step 2/3)
• schannel: encrypted data got 292
• schannel: encrypted data buffer: offset 292 length 4096
• schannel: SSL/TLS handshake complete
• schannel: SSL/TLS connection with storage.googleapis.com port 443 (step 3/3)
• schannel: stored credential handle in session cache

OPTIONS /infovet-9659c.appspot.com/t1587794895719.jpg?X-Goog-Algorithm=GOOG4-RSA-SHA256 HTTP/1.1
Host: storage.googleapis.com
User-Agent: curl/7.55.1
Accept: /
Origin: http://localhost:8100
Access-Control-Request-Method: PUT
Access-Control-Request-Headers: Content-Type

• schannel: client wants to read 102400 bytes
• schannel: encdata_buffer resized 103424
• schannel: encrypted data buffer: offset 0 length 103424
• schannel: encrypted data got 729
• schannel: encrypted data buffer: offset 729 length 103424
• schannel: decrypted data length: 700
• schannel: decrypted data added: 700
• schannel: decrypted data cached: offset 700 length 102400
• schannel: encrypted data buffer: offset 0 length 103424
• schannel: decrypted data buffer: offset 700 length 102400
• schannel: schannel_recv cleanup
• schannel: decrypted data returned 700
• schannel: decrypted data buffer: offset 0 length 102400
  < HTTP/1.1 200 OK
  HTTP/1.1 200 OK
  < X-GUploader-UploadID: AAANsUlPPd2ek77TASSEgsmxUT3BDzEwl1J8rbvSWjkuKr5NTLJP5_jr1c0RJU8YFjRsd7dfBzs1oZoXT1gFvKFQpZM
  X-GUploader-UploadID: AAANsUlPPd2ek77TASSEgsmxUT3BDzEwl1J8rbvSWjkuKr5NTLJP5_jr1c0RJU8YFjRsd7dfBzs1oZoXT1gFvKFQpZM
  < Access-Control-Allow-Origin: *
  Access-Control-Allow-Origin: *
  < Access-Control-Max-Age: 3600
  Access-Control-Max-Age: 3600
  < Access-Control-Allow-Methods: OPTIONS,PUT
  Access-Control-Allow-Methods: OPTIONS,PUT
  < Access-Control-Allow-Headers: Content-Type,X-Requested-With
  Access-Control-Allow-Headers: Content-Type,X-Requested-With
  < Date: Sat, 25 Apr 2020 06:21:49 GMT
  Date: Sat, 25 Apr 2020 06:21:49 GMT
  < Expires: Sat, 25 Apr 2020 06:21:49 GMT
  Expires: Sat, 25 Apr 2020 06:21:49 GMT
  < Cache-Control: private, max-age=0
  Cache-Control: private, max-age=0
  < Content-Length: 0
  Content-Length: 0
  < Server: UploadServer
  Server: UploadServer
  < Content-Type: text/html; charset=UTF-8
  Content-Type: text/html; charset=UTF-8
  < Alt-Svc: quic=":443"; ma=2592000; v="46,43",h3-Q050=":443"; ma=2592000,h3-Q049=":443"; ma=2592000,h3-Q048=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,h3-T050=":443"; ma=259200
  0
  Alt-Svc: quic=":443"; ma=2592000; v="46,43",h3-Q050=":443"; ma=2592000,h3-Q049=":443"; ma=2592000,h3-Q048=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,h3-T050=":443"; ma=2592000

<

• Connection #0 to host storage.googleapis.com left intact
  'X-Goog-Credential' is not recognized as an internal or external command,
  operable program or batch file.
  'X-Goog-Date' is not recognized as an internal or external command,
  operable program or batch file.
  'X-Goog-Expires' is not recognized as an internal or external command,
  operable program or batch file.
  'X-Goog-SignedHeaders' is not recognized as an internal or external command,
  operable program or batch file.
  'X-Goog-Signature' is not recognized as an internal or external command,
  operable program or batch file.
  ````

[febero]

You shouldn't need to include OPTIONS in the config, as it is implied implicitly.

Okay, that makes sense.

If you wouldn't mind confirming this: did your xhr request succeed with the X-Requested-With responseHeader field and failed if it wasn't present?

Yes, exactly.

FWIW, here are the headers of the server response without the X-Requested-With responseHeader

HTTP/2 200 OK
x-guploader-uploadid: AAANsUmaalAEfIy9lPx1o3BeK4x0TLp5GRCrSL1_Sz_jZFJQAyU3pzCnWLFfUUm8Ubzlg_kMesExgHnE3mrY408Nad4
date: Fri, 24 Apr 2020 16:21:13 GMT
expires: Fri, 24 Apr 2020 16:21:13 GMT
cache-control: private, max-age=0
content-length: 0
server: UploadServer
content-type: text/html; charset=UTF-8
alt-svc: quic=":443"; ma=2592000; v="46,43",h3-Q050=":443"; ma=2592000,h3-Q049=":443"; ma=2592000,h3-Q048=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,h3-T050=":443"; ma=2592000

and with X-Requested-With

HTTP/2 200 OK
x-guploader-uploadid: AAANsUl0DXdXi9iJJpleuJq-PPfeeD4LhYT9jGw26SxDhWsSt7mQwxJRTureQ3FB3s95MNnVhGQt77l04FzfZuaVSvJK5XqQ0A
access-control-allow-origin: *
access-control-max-age: 3600
access-control-allow-methods: PUT
access-control-allow-headers: Content-Type,Access-Control-Allow-Origin,X-Requested-With
date: Fri, 24 Apr 2020 16:25:12 GMT
expires: Fri, 24 Apr 2020 16:25:12 GMT
cache-control: private, max-age=0
content-length: 0
server: UploadServer
content-type: text/html; charset=UTF-8
alt-svc: quic=":443"; ma=2592000; v="46,43",h3-Q050=":443"; ma=2592000,h3-Q049=":443"; ma=2592000,h3-Q048=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,h3-T050=":443"; ma=2592000

There was a delta of 4 headers between the two :

access-control-allow-origin: *
access-control-max-age: 3600
access-control-allow-methods: PUT
access-control-allow-headers: Content-Type,Access-Control-Allow-Origin,X-Requested-With

[febero]

@AnuroopSuresh Did you add the Access-Control-Allow-Origin responseHeader too ? I see it's missing in your curl response. Without it my requests were failing too. I should have pointed that out.

[AnuroopSuresh]

@febero Its working, thank you. very much. Do you have any idea why its working in one system and is not working in another system. For instance. now after changing the CORS config, my office system is throwing the error, where as the my personal system is uploading properly. cleared cache history everything in my office system.

[febero]

@febero Its working, thank you. very much. Do you have any idea why its working in one system and is not working in another system. For instance. now after changing the CORS config, my office system is throwing the error, where as the my personal system is uploading properly. cleared cache history everything in my office system.

I noticed that in your first code excerpt you were using a POST method but we then focused on a configuration only allowing PUT. So make sure you've set the actual methods you want to use in your CORS config. You might have mixed them up.

[kikoanis]

After spending few hours on the cors issue, here is what worked for me.

Google Storage Server CORS settings:

     "origin": [<your origins>],
     "responseHeader": ["Content-Type", "Access-Control-Allow-Origin", "X-Requested-With", "x-goog-resumable"],
     "method": ["PUT", "POST"],
     "maxAgeSeconds": 3600

Do not include any thing in your request other than the file buffer/stream in the put signed url request. No authorization header nor any other headers, only the content type (which is file type 'image/jpeg' for images for instance).
Every time you make a change, you need to clear browser cache. That is essential as you may get a previous response.

As chrome is no longer providing access to the options preflight request, you may need to use tools like fiddler or http toolkit to inspect the options preflight response. If the options response does not include Access-Control-Allow-Origin header, it means something not right in your request. Try to correct/use other options till you see that header in the preflight response.

Hope this helps.

[shaffeeullah]

Closing because this seems to be resolved. Please reopen if this is still an issue.