Nodejs-storage: HTTPS_PROXY with HTTP Proxy and google HTTPS URLs does not work.

[refextu]

Environment details

• OS: all
• Node.js version: 8.11.3
• npm version: 5.6.0
• @google-cloud/storage version: 1.7.0
  (1.5.2 is fine)

Steps to reproduce

1. Setup HTTP Proxy
2. Start NodeJS with ENV

HTTPS_PROXY=http://proxy... node script.js

1. upload file to bucket (script.js)

const storage = require('@google-cloud/storage')({
  projectId: 'projectid',
  keyFilename: 'gcloud.json',
});

const bucketname = 'bucketname';
const bucket = storage.bucket(bucketname);

const file = bucket.file('foo');
const stream = file.createWriteStream();
stream.on('error', console.log);
stream.on('finish', console.log);
stream.end('bar');

Information on the root-cause

nodejs-storage uses

• https://github.com/google/google-auth-library-nodejs

which uses gcs-resumable-upload

• https://github.com/stephenplusplus/gcs-resumable-upload

which uses axios and the issue:

• https://github.com/google/google-auth-library-nodejs/issues/352

issues at axios

• https://github.com/axios/axios/issues/925
• https://github.com/axios/axios/issues/1371

pullrequest (open since 1 year caused by bad weather)

• https://github.com/axios/axios/pull/959

[stephenplusplus]

It looks like the axios PR was merged. We probably just need a new release, then we'll all be updated automatically?

cc @JustinBeckwith

[JustinBeckwith]

Working on it :)

@refextu - can you share the specific setup of your upstream proxy? I am struggling to recreate the issue, even though it's obvious there's a problem. Anything you can offer here in the way of a repro would be super helpful.

Also - if I create a patch, would you be willing to try that?

[refextu]

@JustinBeckwith You can use Docker ans Squid for local testing:

docker run --publish 3128:3128 sameersbn/squid:3.3.8-23

I'm happy to test a patch.

[stephenplusplus]

@JustinBeckwith it seems like we have more proxy probs:

• https://github.com/googleapis/nodejs-pubsub/issues/133
• https://github.com/googleapis/nodejs-bigquery/issues/104

Do you think these all chase back to https://github.com/axios/axios/pull/959? There's been some activity on that PR that suggests the fix was not final.

[fhinkel]

@JustinBeckwith was this fixed with the recent auth release?

[JustinBeckwith]

Possibly, maybe, it depends. I'll be a lot more confident once we upgrade to v0.24.0 of nodejs-common.

[stephenplusplus]

We've upgraded! @JustinBeckwith How's the confidence level? 🎚

[JustinBeckwith]

@refextu can you give the 2.x release a shot and let us know if it works?

[stephenplusplus]

@JustinBeckwith if this is fixed, would you suspect https://github.com/googleapis/nodejs-pubsub/issues/133 is fixed as well?

[refextu]

will check later

[refextu]

tested 2.0.3 got error:
write EPROTO 140735840650112:error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol:../deps/openssl/openssl/ssl/s23_clnt.c:827

Test-Code was changed to:

const { Storage } = require('@google-cloud/storage');

const storage = new Storage({
  projectId: 'projectId',
  keyFilename: 'gcloud.json',
});

const bucketname = 'bucketname';
const bucket = storage.bucket(bucketname);

const file = bucket.file('foo');
const stream = file.createWriteStream();
stream.on('error', console.log);
stream.on('finish', console.log);
stream.end('bar');

I think https://github.com/axios/axios/issues/925#issuecomment-419357126 suggests that this will be fixed in 0.19.0 of axios. While current dependency is 0.18.0, as 0.19.0 is not released yet.

[stephenplusplus]

Thanks for the update @refextu.

For now, I'll mark this issue as blocked, as 0.19.0 has still not been released.

[Matt11]

@stephenplusplus - Is this still blocked? Remember there is another issue related to it (https://github.com/googleapis/nodejs-pubsub/issues/133).

[stephenplusplus]

@JustinBeckwith I believe you had been working on the Axios + Proxy issues-- do you know what became of that, and if your prior work would have fixed this?

[JustinBeckwith]

This should be fixed in the latest version of google-auth-library. We implemented a work-around that goes around Axios here:
https://github.com/googleapis/google-auth-library-nodejs/blob/master/src/transporters.ts#L102

I am not the best at setting up or testing proxies :) Would it be possible to set up a proxy server or something in long-door-651 for when we need to debug stuff like this?

[fueltank34]

If it helps, I ended up getting through the proxy by using this
https://www.npmjs.com/package/global-tunnel-ng

[stephenplusplus]

@refextu would you mind testing this again? I haven't been able to reproduce in my testing, but I'm not confident I can mirror your environment accurately enough to promise it has been resolved.

[mainephd]

This appears to still be an issue as of v35.0.0. @stephenplusplus https://github.com/googleapis/nodejs-pubsub/issues/133#issuecomment-443708636

[refextu]

I can confirm, that this is still an issue, but different error:
{ Error: write EPROTO 4584015296:error:1408F10B:SSL routines:ssl3_get_record:wrong version number:../deps/openssl/openssl/ssl/record/ssl3_record.c:252:

[stephenplusplus]

Hello, all! @google-cloud/storage v2.4.0 was released, which hopefully fixes this issue. Would anyone mind testing?

[stephenplusplus]

I'm going to close, but please comment back if things are 👎

[refextu]

Thanks, I can confirm, that this is working like a breeze. 👍