Webpack-dev-server: spdy break content when you use difference protocols for dev server and proxy

[jrweinb]

• Operating System: MacOS High Sierra v 10.13.6
• Node Version: 8.9.2
• NPM Version: 6.4.1
• webpack Version: 4.26.0
• webpack-dev-server Version: 3.1.10

• [x] This is a bug
• [ ] This is a modification request

Code

  // webpack.config.js

config.devServer = {
    hot: true,
    contentBase: resolve(__dirname, '/'),
    disableHostCheck: true,
    publicPath: '/',
    headers: {
        'Accept-Encoding': '',
    },
    historyApiFallback: true,
    host: '0.0.0.0',
    compress: true,
    quiet: true,
    proxy: {
        '/api': {
            target: 'https://XXXXX',
            secure: false,
            changeOrigin: true,
        }
}

I added the Accept-Encoding header per this advice: https://github.com/webpack/webpack-dev-server/issues/354, but it does not solve the problem. I also tried changing the timeout settings without success

Expected Behavior

That when proxying a GET request to /api, the full JSON response will be received

Actual Behavior

Sometimes the request is properly fulfilled and sometimes the response is truncated. This only happens with > 3mb JSON response.

When disabling Chrome web security and making the request directly to the server (without the wds proxy), the request is always fulfilled properly.

For Bugs; How can we reproduce the behavior?

Make a GET request with a JSON response > 3mb through the wds proxy.

To test without the proxy, disable Chrome security and make the request directly to the server.

To disable web security, from terminal:

open /Applications/Google\ Chrome.app --args --user-data-dir="/var/tmp/Chrome dev session" --disable-web-security

[alexander-akait]

@jrweinb can you create minimum reproducible test repo?

[jrweinb]

@evilebottnawi - I was unable to reproduce the error when serving a large file from a local node server. I apologize, but I failed to mention in my bug report that I am on a corporate VPN...I believe that this is the source of the issue. I found a relevant issue in the http-proxy-middleware repo:
https://github.com/chimurai/http-proxy-middleware/issues/171
However, adding a "Connection": "keep-alive" header does not solve the problem for me. The only solution is to run Chrome with web security disabled

[mehl321]

I also got this issue after upgrading from 3.1.10. Big JSON responses get either truncated or even corrupted.

[alexander-akait]

@mehl321 maybe you can create minimum reproducible test repo?

[mehl321]

I created a minimal repo on my machine with a similar config (proxy, HMR, fetching a similar JSON from another local server. No HTTPS though) but I couldn't reproduce the bug.
There is a bug for sure because downgrading to 3.1.10 fixed the issue.

[alexander-akait]

@mehl321 so we can close issue?

[mehl321]

Close the issue if you want but I think the bug is still here. I just didn't manage to reproduce it.

[alexander-akait]

/cc @jrweinb friendly ping

[hkostia]

I'm encountering this too but couldn't make minimum reproducible test repository.

3.1.9 OK
3.1.10+ truncates/corrupts proxied https JSON responses

Operating System: Windows 10
Node Version: 8.12.0
NPM Version: 6.4.1
webpack Version: 4.29.0
webpack-dev-server Version: 3.1.10 - 3.1.14

[alexander-akait]

I think problem on proxy server side, we don't touch content from proxy

[emillynge]

+1 on this issue. downgrade to 3.1.9 worked for me. I have no reproducible test/repo for you.

[alexander-akait]

@eemeli we don't touch anything with 3.1.9 around proxy

[emillynge]

@evilebottnawi Well, it was literally the only change in my setup. Maybe a dependency changed? in which case it would be helpful to track down which dep could cause this.

[alexander-akait]

@emillynge yep, so we need reproducible test repo, it is help to solve this issue

[victors1681]

My Payload is 512KB and I have the same issue. JSON it gets corrupts in webpack-dev-server Version: 3.1.11 - 3.1.14.
The only one is working fine is 3.1.10

[alexander-akait]

@victors1681 please create minimum reproducible test repo, otherwise issue was closed and marked as spam

[edwint229]

I meet some issue in version 3.1.14, my payload is 27.5KB, enabled https proxy. After I downgrade to 3.1.10, it works fine also. I think the issue should happened in proxy side, please help to check and fixed. Thanks

[alexander-akait]

@edwint229 Can you provide minimum reproducible test repo?

[houfeng0923]

i found our api response data (large json ) is truncated . the api server support http2 and it only reproduced in http2 . we can force proxy server use http1.1 always:

```
https: {
key: ...
cert: ...
spdy: {
protocols: ['http1.1']
}
}

// waiting for 'http2' module stable, and express support

[alexander-akait]

@houfeng0923 Can you create minimum reproducible test repo?

[dnicoara-psft]

I'm having the same issue with webpack-dev-server 3.2.1
The JSON response is truncated. Downgrading to 3.1.10 solves the issue.
Could someone take a look at this, please?

[houfeng0923]

@evilebottnawi , i guess a http2 bug in node-spdy or http-proxy-middleware .
webpack-dev-server depend on express , express use node-spdy to support http2 feature .

i ceate a spdy-http2/node-spdy#357 here.

[alexander-akait]

@houfeng0923 thanks for investigateion

[lzane]

same issue here with Node v8
update to Node v10 solve my problem

[yadue]

node v10 forces webpack-dev-server to use https server instead of spdy, I think just an option to select manually would be a solution in that particular case

[alexander-akait]

@yadue why?

[yadue]

for users who are not using nodejs 10 for some reasons it uses only spdy which in webpack-dev-server above 3.1.10 fails for some reasons for the guys here, it truncates response in some cases. there is no way for us to use latest versions of webpack-dev-server since we can't manually select which protocol we want to use

webpack-dev-server detects node v10 and runs https server which works just fine, node < 10, runs spdy library which above version 3.4.7 fails for some users

[alexander-akait]

@yadue it is not solution, we should find problem and fix it, no need implement hacks

[alexander-akait]

What is problem create minimum reproducible test repo?

[yadue]

here you go:
https://github.com/yadue/webpack-dev-server-proxy-issue
npm install
npm run start
open https://app.localhost:4201/
you'll see 2 requests proxied to
http://yadue.synology.me/api3/2jcegul7ephjg2emsfg4v5097g.json (10kb)
http://yadue.synology.me/api3/2jcegul7ephjg2emsfg4v5097g-small.json (2kb)
https://imgur.com/a/sXfyiuY

it really took me so long to find reproducible case.

[alexander-akait]

@yadue very thanks, i will looks on this in near future

[alexander-akait]

@yadue sorry for delay, can't reproduce, can you provide node version? os version? Other information what can help debug problem

[yadue]

Node v8.11.4
Mac OS 10.14.4

[alexander-akait]

I think problem on nodejs/os side, not in webpack-dev-server :disappointed:

[alexander-akait]

Try to reproduce. @yadue just confirm, problem still exists? What workaround you are use?

[alexander-akait]

Reproduced, it is only on node@8, you can use node@10 as workaround

[alexander-akait]

Continue investigate, but looks i find the problem - proxy return output in http1.1, but you run server in http2 mode (we use spdy on node@8 for http2 and looks problem in this module)

[alexander-akait]

Original problem https://github.com/spdy-http2/node-spdy/issues/357. Unfortunately we can fix it right now, it was be fixed after express will be support http2. Now we implement http2 option in master to disable http2, you can use this as workaround

[superchangme]

node v10.16.0 webpack 2.7.1 has the same problem,my heart is crying...

[lennycode]

MeToo. I get truncation or sometimes "holes" in long XML :lol:. I have various micro-services (coors, mixed security, etc).

Guess I will try a newer node.

[superchangme]

hello everybody now i fix this problem,when i try to change every option of my own webpack-dev-server config,when i change compress to false,then everything is ok.
Thank god,god bless us.

config.devServer = {
 ...
 **compress: false,**
 ...
}

[seblegall]

Hi,

Any news on a fix for this bug? I tried to set compress on false but it didn't change anything.

Thx,

[superchangme]

Hi,

Any news on a fix for this bug? I tried to set compress on false but it didn't change anything.

Thx,
my proxyTable is setted on my local nginx server.my local nginx proxy to our backend https api,do you use nginx or use proxyTable to https directly?

[yadue]

@seblegall @superchangme Please try:

        devServer: {
            https: true,
            http2: false,
       }

[seblegall]

👍 It works just fine. thx.

[mice33]

@seblegall @superchangme Please try:

        devServer: {
            https: true,
            http2: false,
       }

👍 It works just fine. thx.

[superchangme]

i find the real reason is my nginx proxy temp file config is no authorization,and i follow this article delete proxy_temp directory，then well done https://blog.csdn.net/e_wsq/article/details/78285185