Nodejs-storage: Using File.download() throws _read() method is not implemented

[segovia-no]

⚠️ BLOCKED

• [x] https://github.com/googleapis/teeny-request/issues/63

This section was added by @stephenplusplus.

---

Environment details

• OS: Windows 10 version 1903
• Node.js version: 10.16.1
• npm version: 6.9.0
• @google-cloud/storage version: 3.0.4

Steps to reproduce

1. Create a new Storage class, use any bucket available in your Google Cloud Dashboard

import {Storage} from '@google-cloud/storage'
gcloudstorage = new Storage({projectId: <yourgcloud_proyectid>, keyFilename: <keyfilenamepath>})

const storage_bucket = gcloudstorage.bucket(<yourgcloud_bucketname>)

1. Inside an async function declare the path variables to get the file and download the file

let filepath = 'company_' + <a_function_parameter> + '_usersignature_' + <another_function_parameter> + '.png'

let anonFilename = Math.random().toString(36).replace('0.', '') //generates a random string
let savePath = __dirname + '../../../downloads/' + anonFilename + '.png'

1. Download the file inside the async block using await

await storage_bucket.file(filepath).download({destination: savePath})

Running these lines you finally get

Error [ERR_METHOD_NOT_IMPLEMENTED]: The _read() method is not implemented
    at Duplex.Readable._read (_stream_readable.js:627:24)
    at Duplex.Readable.read (_stream_readable.js:470:10)
    at resume_ (_stream_readable.js:949:12)
    at process._tickCallback (internal/process/next_tick.js:63:19)

PD: I also verified that the file already exists. tried the 3.0.3 version of @google-cloud/storage with no luck

[BrandiATMuhkuh]

I get suddenly the same error. I'm using firebase storage (which is a wrapper for google cloud storage). A simple re-deploy caused that error. I assume the new version of pumpify https://github.com/googleapis/nodejs-storage/pull/740 causes that.

[vitaliytv]

3.0.4 also same

[ianflorian]

We also experience this issue.

[carnun]

Also experiencing the same issue on a readStream from a file in a bucket (storage.bucket(file.bucket).file(file.name).createReadStream()) on a live google cloud function.
The function does nothing for about 7 seconds and then throws the error, I can't seem to catch the error either.

at process._tickCallback (internal/process/next_tick.js:63:19) 
at resume_ (_stream_readable.js:929:12) 
at Duplex.Readable.read (_stream_readable.js:453:10) 
at Duplex.Readable._read (_stream_readable.js:607:22) 
Error [ERR_METHOD_NOT_IMPLEMENTED]: The _read() method is not implemented stg-bef-upload-data 671771784477209 

Also experienced it locally however, if I revert to v3.0.2 and do a clean install (i.e. rm -rF node_modules + yarn install) it works locally - I still get the issue on GCF.

---update
I reverted to v2.5.0 and my function is working again, hope this helps with debugging the issue.

[thebatt]

I have also suddenly experienced this persistent error since last evening and nothing changed in my code.

[tomphp]

I'm also getting this.

[hadrysmateusz]

I also have the same issue

[kaminskypavel]

working locally with "firebase-emulator" but throws the above error in real cloud functions .. 🤒

[Kyrixx]

Got the same issue there. I've downgraded to 2.5.0 to make my script works

[npomfret]

Ugh, everything works locally, but didn't spot my app was broken in production for hours. How annoying. Anyway, 2.5.0 appears to work on GCP. Also, the error message couldn't be less useful, thanks for spotting @segovia-no

[JustinBeckwith]

@stephenplusplus Here is a minimal repo:

const {Storage} = require('@google-cloud/storage');
const storage = new Storage();
const bucket = storage.bucket('sos-test');
async function main() {
  await bucket.file('test.gif').download({
    destination: './test.gif'
  });
}
main().catch(console.error);

[JustinBeckwith]

Just verified this doesn't repo if you pin @google-cloud/common to 2.0.5. Looks like the 2.1.0 release that went out yesterday is causing the failures.

Talked with @bcoe, the plan is to rollback the dist tag on @google-cloud/common to 2.0.5

[JustinBeckwith]

The rollback of the latest dist tag on @google-cloud/common is complete. Folks on the thread - if you rm -rf node_modules package-lock.json and then npm install again, everything should be working as expected.

I'm downgrading this to a P2 and keeping the issue open to investigate what created the failures in @google-cloud/common version 2.1.0.

[segovia-no]

The rollback of the latest dist tag on @google-cloud/common is complete. Folks on the thread - if you rm -rf node_modules package-lock.json and then npm install again, everything _should_ be working as expected.

I'm downgrading this to a P2 and keeping the issue open to investigate what created the failures in @google-cloud/common version 2.1.0.

Worked like a charm... Thanks

[npomfret]

if you rm -rf node_modules package-lock.json and then npm install again, everything should be working as expected.

Eh? Thanks for the update @JustinBeckwith, but how can this be? Are you saying that I will get different code downloaded from NPM without making any changes to my dependencies?

[kaminskypavel]

@npomfret yes, it looks like 2.1.0 has been removed and 2.0.5 is now the latest,

thanks @JustinBeckwith! that did the trick! 🎉

[npomfret]

@kaminskypavel I know this is off topic, but...

Does this mean that one of my dependencies has a dependency on a latest version (rather than a fixed version number)? So the code I receive from NPM depends entirely on when I run yarn install?

[kaminskypavel]

@npomfret no, npm packages are semversioned and usually use the hat (^) sign in their dependencies.
you can read find more details here

[JustinBeckwith]

So the code I receive from NPM depends entirely on when I run yarn install?

If you have a yarn.lock or package-lock.json, that should give you some promises about which specific versions of dependencies (direct and transitive) are installed. If you remove that lock file, npm or yarn will go through a while process of figuring out based on semver range specifiers which version should get downloaded, and written into the lock file. In the case of a floating range (^ or ~), the latest dist-tag on npm lets the tool know what the "latest" version of the module is going to be. For this fix - I just moved the latest dist-tag back a little, so folks with older lock files, or with no lock file, would get 2.0.5.

I know that's all a mouthful and a little confusing ... but let me know if there's anything I can clear up!

[stephenplusplus]

@bcoe traced the issue back to a change made in our underlying transport library, teeny-request. I opened an issue there so we can look into finding a fix: https://github.com/googleapis/teeny-request/issues/63

Sorry for the trouble, everyone. Let us know if we can help in the meantime.

[bcoe]

this should be fixed in @google-cloud/[email protected].