Azure-sdk-for-js: Problem with passing ProxyOptions to BlobServiceClient

[AZOPCORP]

• Package Name: @azure/storage-blob
• Package Version: 12.1.1
• Operating system: windows 10
• [x] nodejs
  
  
  
  • version:10.16.3
  
  
  • version:
  
  
• Is the bug related to documentation in
  
  
  
  • [x] source code documentation
  
  
  • [x] SDK API docs on https://docs.microsoft.com
  
  

Describe the bug

When I try to specify proxyOptions to a BlobServiceClient created from connectionString it doesn't seems to work, and is not added to the StoragePipeline.

To Reproduce
code:

       const { BlobServiceClient } = require("@azure/storage-blob")
     const connectionString=""
       const blobServiceClient=BlobServiceClient.fromConnectionString(connectionString,{
proxyOptions:
   { host: 'http://localhost',
     port: 80,
     username: 'test',
     password: 'test' }

})
      console.log(blobServiceClient.pipeline)   

when dumping blobServiceClient pipeline i got the following result:

Pipeline {
  factories:
   [ { create: [Function: create] },
     { create: [Function: create] },
     TelemetryPolicyFactory {
       telemetryString:
        'azsdk-js-storageblob/12.1.1 (NODE-VERSION v10.16.3; Windows_NT 10.0.18363)' },
     { create: [Function: create] },
     StorageBrowserPolicyFactory {},
     { create: [Function: create] },
     StorageRetryPolicyFactory { retryOptions: undefined },
     { create: [Function: create] },
     { create: [Function: create] },
     StorageSharedKeyCredential {
       accountName: 'someAccount',
       accountKey:
        <Buffer 9b ... > } ],
  options:
   { proxyOptions: undefined,
     httpClient:
      NodeFetchHttpClient { proxyAgents: {}, keepAliveAgents: {}, cookieJar: [CookieJar] } } }

Expected behavior

when proxyOptions are passed it should be added to blobServiceClient.pipeline.options

Additional context
I am reffering to the following documentations:

https://docs.microsoft.com/en-us/javascript/api/@azure/ms-rest-js/proxysettings?view=azure-node-latest
https://docs.microsoft.com/fr-fr/javascript/api/@azure/storage-blob/storagepipelineoptions?view=azure-node-latest#proxyoptions

[jeremymeng]

@AZOPCORP could you please share the error you got when using proxy? The proxy options are passed into one of the request policy factory.

https://github.com/Azure/azure-sdk-for-js/blob/5cecc379f1c36574b0aaf065d9a94646b97dcb2c/sdk/storage/storage-blob/src/Pipeline.ts#L211

[AZOPCORP]

@jeremymeng No error thrown but proxyblobServiceClient.pipeline.options.proxyOptions is undefined so no way to use proxy.

BTW we found a way to set blobServiceClient.pipeline.options.proxyOptions via the environement variables process.env["HTTPS_PROXY"] and process.env["HTTP_PROXY"]

https://github.com/Azure/azure-sdk/blob/master/_includes/tables/environment_variables.md

exemple:

      process.env["HTTPS_PROXY"]="https://test:test@localhost:80"
      const { BlobServiceClient } = require("@azure/storage-blob")
      const connectionString="xxxxx"
      const blobServiceClient=BlobServiceClient.fromConnectionString(connectionString)

[jeremymeng]

@jeremymeng No error thrown but proxyblobServiceClient.pipeline.options.proxyOptions is undefined so no way to use proxy.

as I mentioned above, in current code the proxy option is already passed to one of the request policy factories. The proxy option will eventually causes the proxyAgent to be used in the node fetch http client when sending requests.

I suspect that the reason why it didn't work for you when passing it via options is that the host properties should not have the http:// part. We only take the host part when getting proxy settings from environment variables. Could you please try again with I was wrong here. We do keep the scheme part (http://).

So I am still curious on why passing proxy settings via option didn't work for you.

[ramya-rao-a]

@jeremymeng Is there anything else we can do here?

[jeremymeng]

@AZOPCORP we don't set blobServiceClient.pipeline.options because it is already passed to the service client via the request policy factories. Please re-open if there are actual issues when connecting to the service using a proxy.

[Marchelune]

Hi,
I face a similar issue to @AZOPCORP. I'm trying to ensure that BlobServiceClient works behind corporate proxies. I'm using Electron (on the main thread) and Fiddler Everywhere to see that the traffic is correctly routed to the proxy.

Doing this, when I use the blobServiceClient to access containers, no traffic shows on Fiddler and it seems to go directly to the internet.

const blobServiceClient = BlobServiceClient.fromConnectionString(connStr, {
    proxyOptions: {
        host: 'http://127.0.0.1',
        port: 8866,
    },
});

Doing it via the env variable on the other hand, I can see traffic on fiddler (at least the connection attempt because Fiddler uses a self-signed certificate for TLS connections, which the underlying node-fetch refuses, but that relates more to this issue)

process.env['HTTP_PROXY'] = 'http://127.0.0.1:8866';
const blobServiceClient = BlobServiceClient.fromConnectionString(connStr);

So to me it seems that the proxyOptions are ignored by the underlying http stack.

Package Name: @azure/storage-blob
Package Version: 12.1.1
Operating system: MacOS Big Sur
nodejs version: 12.12 (electron 8.5)

[jeremymeng]

Yes there is a bug that we would reset options.proxyOptions to empty.

https://github.com/Azure/azure-sdk-for-js/blob/85dcd6ed32108eadf0f023b4acaf6058d113db66/sdk/storage/storage-blob/src/BlobServiceClient.ts#L415

[jeremymeng]

It probably should be

if (!options.proxyOptions) {
  options.proxyOptions = getDefaultProxySettings(extractedCreds.proxyUri); 
}

This would ignore proxyUri in connection string when user passes in proxyOptions via options.
/cc @HarshaNalluru

[EmmaZhu]

Got it. Will fix it.