Nodejs-storage: In the new AppEngine Node.js Standard "resumable: false" is needed

[JustinBeckwith]

_From @juliantoledo on June 14, 2018 8:41_

I just migrated my flex project to AppEngine Node.js Standard and found that "resumable: false" is needed because gcs-resumable-upload tries to write to disk if resumable is enable (this is not allowed in the Standard version)

Steps to reproduce

1. Upload an image similarly to https://github.com/GoogleCloudPlatform/nodejs-getting-started/blob/8bb3d70596cb0c1851cd587d393faa76bfad8f80/3-binary-data/lib/images.js
2. with resumable: enable

Thank you

_Copied from original issue: googleapis/nodejs-datastore#108_

[JustinBeckwith]

@stephenplusplus any thoughts on this one?

[stephenplusplus]

Hello @juliantoledo! What version of nodejs-storage are you using?

[juliantoledo]

Hi @stephenplusplus were are using "@google-cloud/storage": "^1.3.1"

I was able to fix it by adding "resumable: false"

[stephenplusplus]

Thanks. That will definitely work, and as of 1.7, this PR introduced automatically falling back to resumable: false when we can't write to the $HOME directory. However, if the user explicitly asks for a resumable upload by saying resumable: true, we will return an error that lets them know we can't do it.

Does that match your experience?

(Related discussion: https://github.com/googleapis/nodejs-storage/issues/55)

[juliantoledo]

I double checked and we have 1.6.0 in our package-lock.json, that could be the reason because we did not have resumable: true explicitly.

Thanks.

[stephenplusplus]

Thanks for checking. I just did a test with 1.7.0, and found the expected scenarios;

• in GAE standard, without specifying options.resumable, it falls back to a non-resumable upload
• in GAE standard, specifying options.resumable = true, it returns an error: "A resumable upload could not be performed. The directory, /root/.config, is not writable. You may try another upload, this time setting options.resumable to false."

Sounds like we're good here!

[Cdik]

Hi @stephenplusplus , after 2 days of researches, I can't find any answer/explanation so I'm writing here.

How can I then start a resumable upload for files larger than 10MB (as adviced in ResumableUpload documentation)? Here's how I do it:

var options = {
    contentType: "image/jpeg",
    offset: 0,
    resumable: true
  };

file.save(buffer, options, (error, success) => {
    if (!error) {
      return res.status(200).json({
        status: last_chunk ? 200 : 308,
        data: success
      }).send();
    } else {
      return res.status(500).json({
        status: -1,
        data: `Error while saving data: ${error}`
      }).send();
    }
  });

And I get:
ResumableUploadError: A resumable upload could not be performed. The directory, /tmp/.config, is not writable. You may try another upload, this time setting `options.resumable` to `false`.

There's no .config under /tmp and even after changing some folders permission using Cloud Shell, no success, same error.

I'm able to upload files less than 10MB without any problem (with resumable=false) but what about file greater than 10MB?

I also read here that:
Resumable uploads require write access to the $HOME directory. Through config-store, some metadata is stored. By default, if the directory is not writable, we will fall back to a simple upload. However, if you explicitly request a resumable upload, and we cannot write to the config directory, we will return a ResumableUploadError.

But don't understand how to set write access to the $HOME directory. And why config-store should help here.

Desperately waiting for help! Thanks in advance.

[stephenplusplus]

I'm able to upload files less than 10MB without any problem (with resumable=false) but what about file greater than 10MB?

Switch resumable: true to resumable: false.

But don't understand how to set write access to the $HOME directory.

Is that possible, @JustinBeckwith?

And why config-store should help here.

See https://github.com/googleapis/gcs-resumable-upload/#how-it-works.