Yarn: Doesn't work well on a flaky connection

[appsforartists]

Do you want to request a _feature_ or report a _bug_?
Not sure.

What is the current behavior?
I'm at an airport today, so I've been tethering on my phone and using in-flight WiFi. Every time I try to run yarn, I eventually get a socket hang up error on an arbitrary package. Presumably, it doesn't handle mobile connectivity very well. npm install works in these same scenarios.

If the current behavior is a bug, please provide the steps to reproduce.

What is the expected behavior?

• Be able to install a particular module without worrying about the rest of the installed modules.
• Retry requests if one times out.

Please mention your node.js, yarn and operating system version.
yarn v0.15.1
node v6.2.1
macOS v10.11.6

[Daniel15]

Somewhat similar to #627, in that we're not retrying in a case where we probably should.

[designorant]

Interesting. I often work while traveling through areas with little or no coverage, and found using NPM almost impossible then. Yarn seemed very promising, and some quick tests showed it does behave just fine having most/all packages cached, but it does appear equally ineffective when starting from scratch indeed. It would be great if it was more proactive on flaky networks.

---

Context:

I currently work on a project with 803 packages in total.

On a 1mbps, 0% packets drop rate, 5ms delay connection, the install times with cleaned caches are as follows:

• npm install => 8m 0s
• yarn => 3m 38s

On a 1mbps, 10% packets drop rate, 500ms delay connection, the rather unscientific results are as follows:

• npm install just hangs without any feedback at some point.
• yarn trips up either with a socket hang up error, or just hangs.

--verbose was of no use in both cases.

[th3fallen]

can confirm yarn will trip up on a slow/unstable connection with socket hung up regardless of what you do

[deepu105]

I'm facing the same as well. On windows with a tethered mobile connection (not very fast but npm works)

yarn install v0.15.1
info No lockfile found.
[1/4] Resolving packages...
error https://registry.yarnpkg.com/progress: socket hang up
    at createHangUpError (_http_client.js:206:15)
    at TLSSocket.socketOnEnd (_http_client.js:298:23)
    at emitNone (events.js:85:20)
    at TLSSocket.emit (events.js:179:7)
    at endReadableNT (_stream_readable.js:906:12)
    at nextTickCallbackWith2Args (node.js:474:9)
    at process._tickCallback (node.js:388:17)
info Visit http://yarnpkg.com/en/docs/cli/install for documentation about this command.

[juangl]

Same in Archlinux with my 1mbp connection. npm i works.

[gitnik]

This happens to me despite having a very stable connection

[jromero2k]

Windows 10, 512kbps... I can't finish any install. Lots of (random) socket hang ups. NPM works properly most of the time

Example:

yarn add v0.15.1
info No lockfile found.
[1/4] Resolving packages...
error https://registry.yarnpkg.com/babel-preset-es2015-node: socket hang up
at createHangUpError (_http_client.js:252:15)
at TLSSocket.socketOnEnd (_http_client.js:344:23)
at emitNone (events.js:91:20)
at TLSSocket.emit (events.js:185:7)
at endReadableNT (_stream_readable.js:974:12)
at _combinedTickCallback (internal/process/next_tick.js:74:11)
at process._tickCallback (internal/process/next_tick.js:98:9)

[wx-chevalier]

i run into the same problem,in MacOS 10.11.5, node 5.3.0, npm 3.9.5:

 yarn add material-ui
yarn add v0.15.1
info No lockfile found.
warning [email protected]: "dependencies" has dependency "react-komposer" with range "^1.11.0" that collides with a dependency in "devDependencies" of the same name with version "^1.13.1"
[1/4] 🔍  Resolving packages...
error socket hang up
    at TLSSocket.onHangUp (_tls_wrap.js:1090:19)
    at TLSSocket.g (events.js:261:16)
    at emitNone (events.js:73:20)
    at TLSSocket.emit (events.js:167:7)
    at endReadableNT (_stream_readable.js:906:12)
    at nextTickCallbackWith2Args (node.js:455:9)
    at process._tickCallback (node.js:369:17)
info Visit http://yarnpkg.com/en/docs/cli/add for documentation about this command.

[stanonyime]

Having the same issues on Windows 8, yarn 0.15.1, third-world connectivity ( flaky networks). npm works fine

[henri-hulski]

Here exactly the same on Debian Jessie with mobile connection (EDGE about 12kbps).

[henri-hulski]

If I try yarn upgrade I get something like this:

yarn upgrade v0.15.1
[1/4] Resolving packages...
error socket hang up
    at TLSSocket.onHangUp (_tls_wrap.js:1097:19)
    at TLSSocket.g (events.js:273:16)
    at emitNone (events.js:85:20)
    at TLSSocket.emit (events.js:179:7)
    at endReadableNT (_stream_readable.js:913:12)
    at _combinedTickCallback (internal/process/next_tick.js:74:11)
    at process._tickCallback (internal/process/next_tick.js:98:9)
info Visit http://yarnpkg.com/en/docs/cli/upgrade for documentation about this command.

No chance to get it working.

npm works fine.

[george3447]

+1

[hustcer]

yarn v0.16.0
node v6.9.1
macOS v10.12

warning There appears to be trouble with your network connection. Retrying...
warning There appears to be trouble with your network connection. Retrying...
...

however, npm works well.

[mmartinez]

I don't have a flaky connection and I'm having issues as well.

yarn v0.15.1
node v6.9.0
macOS v10.12

> $ yarn add dotenv                                                             [±dev ✓]
yarn add v0.15.1
[1/4] 🔍  Resolving packages...
warning There appears to be trouble with your network connection. Retrying...
warning There appears to be trouble with your network connection. Retrying...
warning There appears to be trouble with your network connection. Retrying...
warning There appears to be trouble with your network connection. Retrying...
warning There appears to be trouble with your network connection. Retrying...
error getaddrinfo ENOTFOUND registry.yarnpkg.com registry.yarnpkg.com:443
    at errnoException (dns.js:28:10)
    at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:76:26)
info Visit http://yarnpkg.com/en/docs/cli/add for documentation about this command.

[RGPaul]

I have the same issue and just found out that Kaspersky Internet Security is blocking the connection from yarn. I don't have problems installing with npm.

[yunxing]

I ran into this issue the other day on VPN, and can reliably repro it until I remove the cache. Maybe it also has something to do with cache.

[PokimLee]

I got the same issue as @mmartinez

[gitnik]

Just investigated this again and for me it's also Kaspersky (on OSX) causing the issue

[jrg3k5]

The same happens to me on 2 mbps line! Npm works fine...

[kvz]

Just hit this but my connection is stable otherwise (I'm posting this without problems - yet consistent failures towards the registry). Also not a kaspersy or other antivirus/proxy user.

[Kitanga]

@gaearon this is the issue I mentioned on twitter

[gaearon]

@Kitanga Thanks. I’m compiling a list in https://github.com/facebookincubator/create-react-app/issues/1189, let me know if you had any other issues.

[caesarsol]

+1, I'm experiencing this as well when the packet drop os high.
Is it possible at least to install as many packages as possible before failing?
(Beside a reasonable number of retries)

[atran]

I fixed it with this local NPM registry DNS comment: https://github.com/npm/npm/issues/6686#issuecomment-246690387

[iraklisg]

Same problem here using Debian Jessie....

warning There appears to be trouble with your network connection. Retrying...
warning There appears to be trouble with your network connection. Retrying...
warning There appears to be trouble with your network connection. Retrying...
warning There appears to be trouble with your network connection. Retrying...
warning There appears to be trouble with your network connection. Retrying...
error An unexpected error occurred: "https://registry.yarnpkg.com/babel-core/-/babel-core-6.23.1.tgz: read ECONNRESET".
info If you think this is a bug, please open a bug report with the information provided in "/var/www/laravel/apptree2/yarn-error.log".
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.

npm works fine with the same setup

[Dakuan]

happening on Heroku this am

[notrab]

Having this problem also on Heroku right now. Deleting yarn.lock would work for those who really need to deploy.

[Dakuan]

magically fixed itself - also had issues with a buildserver failing into install nodejs, glitch in a cdn somewhere?

[drdaeman]

Using vanilla node:7 Docker base image, with Yarn 0.24.4 preinstalled. docker build has RUN yarn install as one of the steps.

I encounter a semi-persistent "There appears to be trouble with your network connection" issue. The actual exception is ESOCKETTIMEDOUT, like this (I've put console.log in onError):

Error:  { Error: ESOCKETTIMEDOUT
    at ClientRequest.<anonymous> (/usr/local/bin/yarn:108470:19)
    at Object.onceWrapper (events.js:293:19)
    at emitNone (events.js:86:13)
    at ClientRequest.emit (events.js:188:7)
    at TLSSocket.emitTimeout (_http_client.js:679:10)
    at Object.onceWrapper (events.js:293:19)
    at emitNone (events.js:86:13)
    at TLSSocket.emit (events.js:188:7)
    at TLSSocket.Socket._onTimeout (net.js:352:8)
    at ontimeout (timers.js:386:14) code: 'ESOCKETTIMEDOUT', connect: false }

I haven't figured out why exactly those fail and why things work fine without Docker. The container has no resource limits (and there are no kernel issues reported) and the network connectivity from the container looked pretty healthy to my probing. For example the latency is low and there is no packet loss neither on small, nor large packets:

--- registry.yarnpkg.com ping statistics ---
1000 packets transmitted, 1000 packets received, 0% packet loss
round-trip min/avg/max/stddev = 11.886/12.664/17.542/0.453 ms

What I've tried and what doesn't seem to help:

1. Forcing isOffline to always return false. Probably irrelevant but I tried this because of this SO answer.

2. Bumping up libuv threadpool size to a maximum with UV_THREADPOOL_SIZE=128. I hoped that would work, because failures could be related to DNS, but it hadn't helped.

3. Passing agent: false and pool: {maxSockets: Infinity} to request module.

Still, "[2/4] Fetching packages..." step retries randomly. Sometimes it passes just fine, sometimes it retries once, sometimes it retries for very long. With --verbose it shows that it downloads most files OK, but sometimes hangs on "Performing "GET" request to ..."

(One of our team members got really unlucky yesterday, and docker build took an hour instead of the usual 2-3 minutes. Could be the case because he ran build on somewhat dated laptop.)

[saravvij]

I am keep getting this error. I have tried at my work, and home network, but no luck.

Environment: Windows 7, yarn 0.27.5, node v7.7.2

Trace: 
  Error: https://registry.yarnpkg.com/caniuse-lite/-/caniuse-lite-1.0.30000697.tgz: ESOCKETTIMEDOUT
      at ClientRequest.<anonymous> (C:\Program Files (x86)\Yarn\lib\yarn-cli.js:114873:19)
      at Object.onceWrapper (events.js:293:19)
      at emitNone (events.js:86:13)
      at ClientRequest.emit (events.js:188:7)
      at TLSSocket.emitTimeout (_http_client.js:679:10)
      at Object.onceWrapper (events.js:293:19)
      at emitNone (events.js:86:13)
      at TLSSocket.emit (events.js:188:7)
      at TLSSocket.Socket._onTimeout (net.js:354:8)
      at ontimeout (timers.js:380:14)

Any solution to this problem?

[temp3l]

adding registry.npmjs.org to /etc/hosts file helped me

in my case:
echo "151.101.12.162 registry.npmjs.org" >> /etc/hosts

[hponiang]

very lucking,thank you @stephanMettler
adding registry.npmjs.org to /etc/hosts file helped me

in my case:
echo "151.101.12.162 registry.npmjs.org" >> /etc/hosts

[BYK]

Is there anything you guys want yarn to do differently? May be add the option to increase retries or something?

Otherwise, I don't see anything actionable here since we cannot fix everyone's internet connection (not yet, anyways).

[dannylewis-sheffield]

OK I had the same issues when building create-react-app inside docker, never managed to install.

Backed out one step to the vagrant box, running on VirtualBox, installed node yarn etc and then again ran create-react-app. Same issue but it did get further, downloaded more packages, than under docker.

Changed the network settings on vagrant from NAT to Bridged:

Old:
config.vm.network :forwarded_port, guest: 80, host: 1026, auto_correct: true

New:
config.vm.network :public_network, :public_network => "wlan0"

You can switch back the configs once you have successfully done what you needed to.

Ran create-react-app again and it looked like it it would complete, despite the odd "warning There appears to be trouble with your network connection. Retrying..." Nope failed on ESOCKETTIMEDOUT, but just about there!

Ran it again, success!

Back to docker version, long story short it eventually successfully built. Good job I'm doing intermediate builds...

Networks can be flaky, some more than others.

I would like to see the following be able to be set in config:

download_max_retry - does what it says on the tin and defaults to 20
download_clear_failed_count_on_success - true or false, default false, if true would reset the failed download count to 0 on the next successful download, this would mean it would only fail to complete if it failed to download the last file in the queue (assuming downloads are parallel, some fail and are then put back in the queue)

Hope this helps and thanks for the tools!

[davidcalhoun]

Getting a similar situation today. I'm running a yarn upgrade command and got this message multiple times:

warning There appears to be trouble with your network connection. Retrying...

Then got

error An unexpected error occurred: "https://registry.yarnpkg.com/material-ui: getaddrinfo ENOTFOUND registry.yarnpkg.com registry.yarnpkg.com:443".

info If you think this is a bug, please open a bug report with the information provided in "/Users/davidcalhoun/dev/curator/yarn-error.

These were caused by connection issues, but once they resolved themselves I found myself unable to run the command for a few minutes. Running the command from a new terminal window instantly worked.

[BYK]

These all look like DNS issues. I do not know why yarn saturates the DNS resolving though. I'll try to investigate this.

[MichelDiz]

Does look like a DNS issue. I have this issue using Rancher (that has complicated workarounds with DNS).

UPDATE: Solved putting the stacks to use network the "bridge" - For me it is a temporary solution, because my environment is for tests.

Inside the app that is inside the the Rancher environment

/usr/src/app # node -e 'const os = require("os"); const interfaces = os.networkInterfaces(); for (const interface in interfaces) {console.log(interface); const addrs = interfaces[interfac
e]; for (const addr of addrs) {console.log(addr.address)}}'
lo
127.0.0.1
eth0
10.42.222.77

cat hosts
127.0.0.1       localhost
::1     localhost ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters

10.42.222.77    441206ad059d

Inside the app that is inside the the Rancher environment

/usr/src/app # ping yarn.org
PING yarn.org (184.168.221.96): 56 data bytes
64 bytes from 184.168.221.96: seq=0 ttl=61 time=222.452 ms
64 bytes from 184.168.221.96: seq=1 ttl=61 time=225.904 ms
64 bytes from 184.168.221.96: seq=2 ttl=61 time=223.484 ms
64 bytes from 184.168.221.96: seq=3 ttl=61 time=222.596 ms
64 bytes from 184.168.221.96: seq=4 ttl=61 time=222.305 ms
64 bytes from 184.168.221.96: seq=5 ttl=61 time=221.858 ms
64 bytes from 184.168.221.96: seq=6 ttl=61 time=222.100 ms
64 bytes from 184.168.221.96: seq=7 ttl=61 time=222.088 ms
64 bytes from 184.168.221.96: seq=8 ttl=61 time=221.959 ms

Inside the app that is inside the the Rancher environment

/usr/src/app # nslookup registry.npmjs.org
nslookup: can't resolve '(null)': Name does not resolve

Name:      registry.npmjs.org
Address 1: 151.101.176.162

Outside of Rancher (in docker machine)

[margusha]

Is there anything you guys want yarn to do differently? May be add the option to increase retries or something?

Otherwise, I don't see anything actionable here since we cannot fix everyone's internet connection (not _yet_, anyways).

100mbps down, 26mbps up current internet speed test. ping 7ms. doesn't seem like a network issue. but yet again, yarn install and yarn start fails with ETIMEDOUT and it has done it randomly for a past year. running npm install starts up the app in ~10 seconds. Guess it was my last time to use yarn.