Nodejs-storage: Cannot insert legacy ACL for an object when uniform bucket-level access is enabled

[Cherviakov]

Thanks for stopping by to let us know something could be better!

PLEASE READ: If you have a support contract with Google, please create an issue in the support console instead of filing on GitHub. This will ensure a timely response.

1) Is this a client library issue or a product issue?
This is the client library for . We will only be able to assist with issues that pertain to the behaviors of this library. If the issue you're experiencing is due to the behavior of the product itself, please visit the Support page to reach the most relevant engineers.

2) Did someone already solve this?

• Search the issues already opened: https://github.com/googleapis/nodejs-storage/issues
• Search the issues on our "catch-all" repository: https://github.com/googleapis/google-cloud-node
• Search or ask on StackOverflow (engineers monitor these tags): http://stackoverflow.com/questions/tagged/google-cloud-platform+node.js

3) Do you have a support contract?
Please create an issue in the support console to ensure a timely response.

If the support paths suggested above still do not result in a resolution, please provide the following details.

Environment details

• OS: linux ubuntu 16.04
• Node.js version: 14.4.0
• npm version: 6.14.15
• @google-cloud/storage version: 5.1.1

Steps to reproduce

1. On google cloud create bucket and configure uniform bucket-level access
2. Instantiate Storage and specify bucket
3. fs.createReadableStream(path.resolve('myFile.txt')).pipe(bucket.file('test').createWritableStream({ contentType: 'text/plain' }))

4. Get 400 error saying Cannot insert legacy ACL for an object when uniform bucket-level access is enabled

   So I tried to look around for solution and dig into refernece of nodejs client lib, but cannot find way to avoid sending predefinedAcl, which I think is problem here. If I set predefinedAcl to null or empty string then it get default value. Also I cannot
   find any example of code, which explains how to work with uniform bucket-level access (aside of examples how to switch it on/off and get metadata value to know if it is switched on).

Making sure to follow these steps will guarantee the quickest resolution possible.

Thanks!

[jkwlui]

Thank you for reporting the issue, @Cherviakov.

I've tried your reproduction steps on a uniform bucket-level access-enabled bucket but have failed to reproduce the issue you described. Can you provide the full code that reproduces the problem?

[Cherviakov]

const fs = require('fs');
const path = require('path');
const { Storage } = require('@google-cloud/storage');
const { client_email, private_key } = require('../../config');

class GoogleStorageService {
  constructor () {
    this.storage = new Storage({
      credentials: {
        client_email,
        private_key
      }
    })
  }

  getUploadWritableStream (fileName, contentType) {
    return this.bucket.file(fileName).createWritableStream({
      contentType,
    })
  }
}

class GoogleStorageMy extends GoogleStorageService {
  constructor () {
    super();
    this.bucket = this.storage.bucket('my-bucket');
  }
}

const service = new GoogleStorageMy;
const stream = fs.createReadStream(path.resolve('my.pdf')).pipe(service.getUploadWritableStream('test', 'application/pdf'));
stream.on('error', console.error);
stream.on('finish', () => console.log('done'));

Additional notes: I do not have direct access to cloud, devops working there separately, so he created bucket and made some updates, after which I tried to upload file with abovementioned code and get error about inserting ACL.
To this moment all I tried shows that this code should be working pretty same with uniform access as with ACL (before worked with ACL with another old bucket), therefore I start to believe that maybe this problem related to some errors in configuration of bucket itself? But in this case error 400 looks strange, as it usually means that client messed up with something and sending request not correctly.

[jkwlui]

Thanks for adding your code!

I don't see anything from there that would cause ACLs to be set, which triggers this error message. According to the docs:

• Requests to set, read, or modify bucket and object ACLs fail with 400 Bad Request errors.
  
  
  
  • This includes JSON API requests using any BucketAccessControls, DefaultObjectAccessControls, or ObjectAccessControls methods.
  
  

Can you try adding NODE_DEBUG=https to your environment and verify if the predefinedAcl query string is sent along with your upload request? (take care to make sure your auth tokens or bucket/object names are not leaked).

[Cherviakov]

Sadly error logs not saved and that bucket was removed, but I definitely remember that predefinedAcl was sent in query params. I also checked code of client library and looks that predefinedAcl would be always present, as it use default value even if user not set it, and I couldn't find option nor in docs not in source code of client library, how to ask not to send it.

Seems I can't really provide more info on this issue as decision made to use old ACL access, and I can't test that problem anymore.

[wentjun]

This is strange. For some reason, I am getting the very same error when I migrated to use uniform access. As you can see, I have removed the options when I call createWriteStream(),

const stream = doc.pipe(file.createWriteStream());

However, this seems to throw an error with the following error:

{\"message\":\"Cannot insert legacy ACL for an object when uniform bucket-level access is enabled. Read more at https://cloud.google.com/storage/docs/uniform-bucket-level-access.\",\"domain\":\"global\",\"reason\":\"invalid\"}

Upon inspecting the full error message, it seems like predefinedAcl=projectPrivate was appended to the request URL? I am not sure why it is there even though I clearly did not include any of the options. Could this be causing the issue?

[vanhumbeecka]

Same problems here. Reopen this issue please.

I've tried some settings, like expicitaly settin predefinedAcl: undefined without success. Still getting the same error responses. The returned URL's do not help at all.

{ code: 400,
  message:
   'Cannot insert legacy ACL for an object when uniform bucket-level access is enabled. Read more at https://cloud.google.com/storage/docs/uniform-bucket-level-access',
  errors:
   [ { message:
        'Cannot insert legacy ACL for an object when uniform bucket-level access is enabled. Read more at https://cloud.google.com/storage/docs/uniform-bucket-level-access',
       domain: 'global',
       reason: 'invalid' } ] }

[icoloma]

To those that make the same mistake as me, remember to remove public: true from your options.

[pvgn]

I switched to uniform access control, and strangely, I have that error on 1 file.
It's a simple file I called version at the root of the bucket.
It's just impossible to write there anymore 🤷🏻‍♂️
I still can write to any other file, except that one...
The only conf I used is that :

file.createWriteStream({ metadata: { contentType: 'text/plain' }})

My workaround is just to rename the file...

[vipien]

I am having the same error, I am using version 5.5.0.

I created the Storage

  this.storage = new Storage({
      keyFilename: './key.json',
      email: '[email protected]',
      projectId: xxxxxxx
    })

And I upload the file

 await new Promise((resolve, reject) => {
      stream
        .on('error', err => {
          console.log('err1', err)
          reject
        })
        .pipe(
          this.storage
            .bucket(`user-${this.user.id}`)
            .file(`${filename}`)
            .createWriteStream()
            .on('error', err2 => {
              console.log('err2', err2)
            })
            .on('finish', resolve)
        )
    })

I tried adding predefinedAcl as false and null but it does not work. Any suggestion?

Thanks

[btilford]

So I think this happens because of the resumable upload it's doing behind the scenes. If your first attempt at uploading a file fails then you remove the ACL it's not actually re-creating the URI it's loading it from ~/.config/configstore/gcs-resumable-upload.json.

I feel like this is a bug in the Configstore where it's not rebuilding the entire URI and it's not detecting that you have removed the ACL.

[AllNamesRTaken]

I have the same issue and the only workaround was to remove the resumable upload.