Node: highWaterMark option is ignored in HTTPs request stream

Created on 11 Apr 2020  路  5Comments  路  Source: nodejs/node

Hi,

https request stream ignores the custom highWaterMark size option. In http request it works perfectly.

Version: All of them, even nightly
Platform: Windows 10 64-bit
Platform: https

What steps will reproduce the bug?

Execute the following code snippet using node

var https = require('https');

var url = "https://www.facebook.com";      
var options = { rejectUnauthorized: false, 
                highWaterMark: 128000  };

var request = https.request(url, options, function(response)   {
     // it returns 16384 with https but 128000 in http
     console.log("HTTPS readableHighWaterMark: " + response.readableHighWaterMark);  
}).end();

var http = require('http');

var url = "http://www.facebook.com";
var options = {  highWaterMark: 128000  };

var request = http.request(url, options, function(response)   {
     // it returns 16384 with https but 128000 in http
     console.log("HTTP readableHighWaterMark: " + response.readableHighWaterMark);  
}).end();

How often does it reproduce? Is there a required condition?

No. This potential bug can always be reproduced.

### What is the expected behavior?
https stream readableHighWaterMark should be 128000 instead 16384.

What do you see instead?

Console Output:

HTTP readableHighWaterMark: 128000
HTTPS readableHighWaterMark: 16384

All 5 comments

I'm try working on this.

There is still something strange...

Testing with Node.js v15.0.0-nightly20200505c17dcb3253 it returns fine now the highWaterMark in https module but it is still downloading with chunks of 16Kb

HTTPS Test
--------------------------------------------------
Response.readableHighWaterMark: 131072
Outfile WriteStream WritableHighWaterMark: 131072

Progress: 0%    16.00 kB / 21.85 MB
Progress: 0%    32.00 kB / 21.85 MB
Progress: 0%    48.00 kB / 21.85 MB
Progress: 0%    64.00 kB / 21.85 MB
Progress: 0%    80.00 kB / 21.85 MB
Progress: 0%    96.00 kB / 21.85 MB
Progress: 0%    112.00 kB / 21.85 MB
...

Testing the same file to download, to the same server in a LAN with http instead https we are receiving in "on data" callback bigger chunks

HTTP Test
--------------------------------------------------
Response.readableHighWaterMark: 131072
Outfile WriteStream WritableHighWaterMark: 131072

Progress: 0%    10.69 kB / 21.85 MB
Progress: 0%    74.69 kB / 21.85 MB
Progress: 0%    138.69 kB / 21.85 MB
Progress: 0%    202.69 kB / 21.85 MB
Progress: 1%    266.69 kB / 21.85 MB
Progress: 1%    330.69 kB / 21.85 MB
Progress: 1%    394.69 kB / 21.85 MB
Progress: 2%    458.69 kB / 21.85 MB
Progress: 2%    522.69 kB / 21.85 MB
Progress: 2%    586.69 kB / 21.85 MB
...

Should I fill a new ticket?

I'm trying to replicate this problem.

@thunderol could you create a separate GH issue? It'll be easier to track the fix this way.

Created new issue #33262

Was this page helpful?
0 / 5 - 0 ratings

Related issues

stevenvachon picture stevenvachon  路  3Comments

cong88 picture cong88  路  3Comments

danialkhansari picture danialkhansari  路  3Comments

willnwhite picture willnwhite  路  3Comments

seishun picture seishun  路  3Comments