Hi,
https request stream ignores the custom highWaterMark size option. In http request it works perfectly.
Version: All of them, even nightly
Platform: Windows 10 64-bit
Platform: https
Execute the following code snippet using node
var https = require('https');
var url = "https://www.facebook.com";
var options = { rejectUnauthorized: false,
highWaterMark: 128000 };
var request = https.request(url, options, function(response) {
// it returns 16384 with https but 128000 in http
console.log("HTTPS readableHighWaterMark: " + response.readableHighWaterMark);
}).end();
var http = require('http');
var url = "http://www.facebook.com";
var options = { highWaterMark: 128000 };
var request = http.request(url, options, function(response) {
// it returns 16384 with https but 128000 in http
console.log("HTTP readableHighWaterMark: " + response.readableHighWaterMark);
}).end();
No. This potential bug can always be reproduced.
### What is the expected behavior?
https stream readableHighWaterMark should be 128000 instead 16384.
Console Output:
HTTP readableHighWaterMark: 128000
HTTPS readableHighWaterMark: 16384
I'm try working on this.
There is still something strange...
Testing with Node.js v15.0.0-nightly20200505c17dcb3253 it returns fine now the highWaterMark in https module but it is still downloading with chunks of 16Kb
HTTPS Test
--------------------------------------------------
Response.readableHighWaterMark: 131072
Outfile WriteStream WritableHighWaterMark: 131072
Progress: 0% 16.00 kB / 21.85 MB
Progress: 0% 32.00 kB / 21.85 MB
Progress: 0% 48.00 kB / 21.85 MB
Progress: 0% 64.00 kB / 21.85 MB
Progress: 0% 80.00 kB / 21.85 MB
Progress: 0% 96.00 kB / 21.85 MB
Progress: 0% 112.00 kB / 21.85 MB
...
Testing the same file to download, to the same server in a LAN with http instead https we are receiving in "on data" callback bigger chunks
HTTP Test
--------------------------------------------------
Response.readableHighWaterMark: 131072
Outfile WriteStream WritableHighWaterMark: 131072
Progress: 0% 10.69 kB / 21.85 MB
Progress: 0% 74.69 kB / 21.85 MB
Progress: 0% 138.69 kB / 21.85 MB
Progress: 0% 202.69 kB / 21.85 MB
Progress: 1% 266.69 kB / 21.85 MB
Progress: 1% 330.69 kB / 21.85 MB
Progress: 1% 394.69 kB / 21.85 MB
Progress: 2% 458.69 kB / 21.85 MB
Progress: 2% 522.69 kB / 21.85 MB
Progress: 2% 586.69 kB / 21.85 MB
...
Should I fill a new ticket?
I'm trying to replicate this problem.
@thunderol could you create a separate GH issue? It'll be easier to track the fix this way.
Created new issue #33262