Apm: `apm outdated` and `apm upgrade` don't work for me; other commands work fine

Created on 6 Nov 2016 · 21Comments · Source: atom/apm

apm outdated and apm upgrade are returning with a ETIMEDOUT error.

I initially thought the issue may have been a network issue on my part, but apm search <package> and apm install <package> work just fine. For now, I'm upgrading packages by using apm install <package>

Reproduction Steps:

Run apm outdated or apm upgrade <optional-package-name>

Expected behavior:

List of outdated packages or begin upgrading outdated packages.

Observed behavior:

After no output for several seconds:

apm outdated shows error Request for package information failed: connect ETIMEDOUT 54.243.164.241:443
apm upgrade shows error Request for package information failed: read ETIMEDOUT

Versions (reported by apm):

apm 1.12.5
npm 3.10.5
node 4.4.5
python 2.7.11
git 2.8.4

Atom version: 1.11.2
OS and version: macOS 10.12

Installed packages: will include if necessary on separate post

Additional information:

Problem can be reliably reproduced, doesn't happen randomly: Has been reliably reproduced for last couple of days

bug needs-reproduction

Source

blakek

Most helpful comment

Due to #665 I just took a look at upgrade.coffee or more so the final upgrade.js and did some testings.

https://github.com/atom/apm/blob/master/src/upgrade.coffee#L80 is the line where the problems begin.

I wrote a little testing script which requests the same package 100 times:

var request = require('request');

var requestSettings = {
    url: 'https://atom.io/api/packages/minimap',
    json: true
};

for(var i=0; i<100; i++) {

    request.get(requestSettings, function(error, response, body) {
            console.log(error)
        }
    );
}

The first 40 requests went through without any problems. All of the other 60 failed:

{ Error: connect ETIMEDOUT 54.225.223.184:443
    at Object.exports._errnoException (util.js:1022:11)
    at exports._exceptionWithHostPort (util.js:1045:20)
    at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1087:14)
  code: 'ETIMEDOUT',
  errno: 'ETIMEDOUT',
  syscall: 'connect',
  address: '54.225.223.184',
  port: 443 }

Sidenote: Returning an empty callback function in https://github.com/atom/apm/blob/master/src/upgrade.coffee#L81 lets me update at least some of my packages.

So in my opinion there are three possibilities:

Because all requests for all packages seem to be fired asynchronously at once we might hit a limit configured in the webserver which sets us on hold after 40 concurrent connections and thus triggers an timeout?
While it does not seem to be a problem on unix systems we might hit a connection limit on Windows?
There is some serious f*#$up in the requests lib...

I think the behaviour of getAvailableUpdates should be changed. Instead of mapping all packages to async and firing all requests at once, which itself is a bad idea because you should not fire hundreds of requests at once, you should consider doing them one after another. Sure it will be slower but it would be a much cleaner and safer way - just my two cents ;)

bennyborn on 31 Dec 2016

👍2

All 21 comments

Also, the results of apm config list:

; cli configs
globalconfig = "/Users/blake/.atom/.apm/.apmrc"
user-agent = "npm/3.10.5 node/v4.4.5 darwin x64"
userconfig = "/Users/blake/.atom/.apmrc"

; environment configs
node-gyp = "/Applications/Atom.app/Contents/Resources/app/apm/bin/../node_modules/.bin/node-gyp"

; project config /Users/blake/.npmrc
depth = 0
init-author-email = "[email protected]"
init-author-name = "Blake Knight"
init-author-url = "http://blakek.me/"
init-license = "MPL-2.0"
unicode = true
viewer = "browser"

; globalconfig /Users/blake/.atom/.apm/.apmrc
cache = "/Users/blake/.atom/.apm"
progress = false

; node bin location = /Applications/Atom.app/Contents/Resources/app/apm/bin/node
; cwd = /Users/blake
; HOME = /Users/blake/.atom/.node-gyp
; "npm config ls -l" to show all defaults.

blakek on 6 Nov 2016

This seems to be the exact issue I've been having on Windows. I can't update from Atom and apm update, upgrade and outdated return the same error:
Request for package information failed: connect ETIMEDOUT 23.21.228.240:443
I booted into Ubuntu to check if this has to to with my network but I had no problems there.

Versions (reported by apm):
apm 1.12.9
npm 3.10.5
node 4.4.5
python 2.7.12
git 2.10.0.windows.1
visual studio

Atom version: 1.12.6
OS and version: Windows 10 64-bit

Thanks @blakek for the apm install <package> idea. I haven't thought of that and I've been running Atom with outdated packages until now.

chiarpavel on 6 Dec 2016

Due to #665 I just took a look at upgrade.coffee or more so the final upgrade.js and did some testings.

https://github.com/atom/apm/blob/master/src/upgrade.coffee#L80 is the line where the problems begin.

I wrote a little testing script which requests the same package 100 times:

var request = require('request');

var requestSettings = {
    url: 'https://atom.io/api/packages/minimap',
    json: true
};

for(var i=0; i<100; i++) {

    request.get(requestSettings, function(error, response, body) {
            console.log(error)
        }
    );
}

The first 40 requests went through without any problems. All of the other 60 failed:

{ Error: connect ETIMEDOUT 54.225.223.184:443
    at Object.exports._errnoException (util.js:1022:11)
    at exports._exceptionWithHostPort (util.js:1045:20)
    at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1087:14)
  code: 'ETIMEDOUT',
  errno: 'ETIMEDOUT',
  syscall: 'connect',
  address: '54.225.223.184',
  port: 443 }

Sidenote: Returning an empty callback function in https://github.com/atom/apm/blob/master/src/upgrade.coffee#L81 lets me update at least some of my packages.

So in my opinion there are three possibilities:

Because all requests for all packages seem to be fired asynchronously at once we might hit a limit configured in the webserver which sets us on hold after 40 concurrent connections and thus triggers an timeout?
While it does not seem to be a problem on unix systems we might hit a connection limit on Windows?
There is some serious f*#$up in the requests lib...

bennyborn on 31 Dec 2016

👍2

@bennyborn thanks for your research!

I am seeing the same errors on several macOS systems from different areas/different ISPs. The first possibility seems the most probable cause to me. Also, maybe apm install is working because it's usually searching for one package at a time?

blakek on 31 Dec 2016

I've got the same issue running Win10 64bit and Atom 1.13.0. Thanks to @blakek for the workaround with updating each package individually.

For what it is worth: I decided to try out the beta version 1.14.0-beta0 by downloading and running the beta atomsetup.exe. The setup launches atom automatically and I noticed that it is able to make a single successful check for updates before the ETIMEDOUT error starts (this is from the settings panel not command line). I was able to reproduce this behavior three times by reinstalling the 1.13.0 atomsetup.exe then running the beta atomsetup.exe.

joshuadavidson on 13 Jan 2017

What's the status of this issue? I've just started experiencing this issue recently (I don't think it was doing this in 0.12....?). I'm running atom 0.13.1ia32 (although my machine is 64 bit.....? Wat?)

sbrl on 26 Jan 2017

x64 builds are only available for 1.14.0 and higher.

There has been no progress on this issue, mainly because no one from the team has managed to reproduce it.

50Wliu on 26 Jan 2017

@50Wliu Is there really a need to reproduce this? I think the main problem is quite obvious. Trying to requests data for all packages at once which could possibly result in hundreds of connections at a single time.

Add a limit of maybe 10 concurrent connections and I guess everyone of us who is affected will be fine.

bennyborn on 26 Jan 2017

@50Wliu In which case I have 1.13.1 - I misread it! I'll look into upgrading to the _actual_ latest version asap - that should fix the issue I should imagine. I'll report back either way.

@bennyborn That sounds like a sensible idea. Limiting the concurrent connections should increase overall stability. Perhaps a queue of sorts would work?

sbrl on 26 Jan 2017

@sbrl If memory serves me correctly they're using the async module to map all packages to an update function. If this is the case it's just a matter of replacing map with mapLimit and voila they'll have a queue of n concurrent connections :)

bennyborn on 26 Jan 2017

@bennyborn sounds like you have it all figured out. Perhaps you could submit a PR and test it for us? Since you're affected you would be able to verify that it fixes the issue. We aren't and haven't been able to reproduce the problem, so we can't verify for certain that any code change would actually fix the problem you're experiencing.

lee-dohm on 26 Jan 2017

Erm oops! 1.14 is the beta version, and I'm on stable! In which case for 1.14 I'll wait until release.

sbrl on 27 Jan 2017

While it worked once or twice, it's now stopped working again. I'm now on 1.15.0 - same error message:

Request for package information failed: connect ETIMEDOUT 184.72.248.171:443

sbrl on 20 Mar 2017

same here

Request for package information failed: connect ETIMEDOUT 23.23.223.197:443

bonesoul on 10 Apr 2017

λ nslookup atom.io
Server:  google-public-dns-a.google.com
Address:  8.8.8.8

Non-authoritative answer:
Name:    atom.io
Addresses:  23.23.128.123
          23.23.223.197
          50.19.93.247
          23.23.117.228
          54.225.186.255
          184.72.248.171
          23.21.77.86
          54.235.135.158

tried all these through my hosts file but i still can't get this fixed.

bonesoul on 12 Apr 2017

small update in case anyone is still looking into this:
It's still not working for me, same symptoms, but I've found that if I'm using a different ISP everything works (I used my phone as a hotspot and I can't believe I haven't tried this until now).
This leads me to believe that the problem is related to the ISP.

chiarpavel on 16 May 2017

I have connections of different ISPs, one at home & a different provider at the office. Same symptoms. My gut feeling is that this has more to do with the request hammering than anything else.

jagged3dge on 16 May 2017

@chiarpavel I've seen similar issues where switching ISPs will sometimes cause correct behavior. However, I've also seen where an ISP that once worked will stop working (with our internal DNS settings staying constant). I believe @jagged3dge is correct—either apm or the package servers are not gracefully handling multiple requests, which is why I believe apm install and apm search still work properly.

blakek on 19 May 2017

Since #677 was merged, I have noticed this happening _much_ less often, but it does still seems to happen once in a while. Unfortunately I have not been able to pin down exact cases where I can consistently reproduce.

blakek on 19 May 2017

Atom v1.17.0, still happening, been happening for years:

Request for package information failed: connect ECONNREFUSED 74.195.29.38:443

Yet individual updating works.

OvermindDL1 on 27 May 2017

Same as above still a problem in v1.17.0.

This may not be a problem with the Atom / apm client.
This could be a problem with the infrastructure handling this request.
Has anybody checked that out what is actually happening with these requests on the servers?

I did this over about an hour to finally get all the packages updated.
I am connecting from South Africa so it does take me a lot longer to actually get to the USA.

aaron@aaron-pc: ~ 
$ tracepath 50.16.231.196
 1?: [LOCALHOST]                      pmtu 1500
 1:  gateway                                               0.800ms 
 1:  gateway                                               0.751ms 
 2:  192.168.0.9                                           0.938ms 
 3:  41.221.9.97                                           1.140ms 
 4:  10.251.0.134                                          2.578ms 
 5:  10.251.0.133                                          2.147ms 
 6:  rrba-ip-ll-1-wan.telkom-ipnet.co.za                   2.780ms 
 7:  lon-ip-hsll-1-gig-5-1-0.telkom-ipnet.co.za          166.830ms asymm 11 
 8:  xe-0-1-0-19.r00.londen10.uk.bb.gin.ntt.net          183.333ms asymm 15 
 9:  ae-13.r24.londen12.uk.bb.gin.ntt.net                186.072ms asymm 15 
10:  ae-5.r24.nycmny01.us.bb.gin.ntt.net                 252.239ms asymm 16 
11:  ae-1.r08.nycmny01.us.bb.gin.ntt.net                 245.776ms asymm 15 
12:  ae-11.amazon.nycmny01.us.bb.gin.ntt.net             250.832ms asymm 17 
13:  no reply
^C

aaron@aaron-pc: ~ 
$ tracepath 50.16.192.69
 1?: [LOCALHOST]                      pmtu 1500
 1:  gateway                                               0.788ms 
 1:  gateway                                               0.761ms 
 2:  192.168.0.9                                           0.860ms 
 3:  41.221.9.97                                           0.933ms 
 4:  10.251.0.134                                          2.284ms 
 5:  10.251.0.133                                          2.135ms 
 6:  rrba-ip-ll-1-wan.telkom-ipnet.co.za                   2.523ms 
 7:  lon-ip-hsll-1-gig-5-1-0.telkom-ipnet.co.za          166.274ms asymm 11 
 8:  xe-0-1-0-19.r00.londen10.uk.bb.gin.ntt.net          181.485ms asymm 15 
 9:  ae-13.r24.londen12.uk.bb.gin.ntt.net                182.897ms asymm 15 
10:  ae-5.r24.nycmny01.us.bb.gin.ntt.net                 247.089ms asymm 16 
11:  ae-1.r08.nycmny01.us.bb.gin.ntt.net                 254.050ms asymm 15 
12:  ae-10.amazon.nycmny01.us.bb.gin.ntt.net             257.307ms asymm 17 
13:  no reply

$ apm upgrade
Package Updates Available (7)
├── atom-beautify 0.29.24 -> 0.29.26
├── busy-signal 1.4.1 -> 1.4.3
├── linter-json-lint 0.1.1 -> 0.1.2
├── linter-pyflakes 0.3.1 -> 0.3.2
├── linter-pylint 2.0.1 -> 2.1.0
├── linter-ui-default 1.2.4 -> 1.6.0
└── vim-mode-plus 0.91.0 -> 0.93.0

Would you like to install these updates? (yes)

Installing [email protected] to /home/aaron/.atom/packages ✓
Installing [email protected] to /home/aaron/.atom/packages ✓
Installing [email protected] to /home/aaron/.atom/packages Unable to download https://www.atom.io/api/packages/linter-json-lint/versions/0.1.2/tarball: connect ETIMEDOUT 50.16.192.69:443

$ apm upgrade
Package Updates Available (5)
├── linter-json-lint 0.1.1 -> 0.1.2
├── linter-pyflakes 0.3.1 -> 0.3.2
├── linter-pylint 2.0.1 -> 2.1.0
├── linter-ui-default 1.2.4 -> 1.6.0
└── vim-mode-plus 0.91.0 -> 0.93.0

Would you like to install these updates? (yes)

Installing [email protected] to /home/aaron/.atom/packages ✓
Installing [email protected] to /home/aaron/.atom/packages ✓
Installing [email protected] to /home/aaron/.atom/packages Unable to download https://www.atom.io/api/packages/linter-pylint/versions/2.1.0/tarball: connect ETIMEDOUT 50.16.214.206:443

$ apm upgrade
Request for package information failed: connect ETIMEDOUT 50.16.231.196:443

$ apm upgrade
Request for package information failed: connect ETIMEDOUT 50.17.199.1:443

$ apm upgrade
Request for package information failed: connect ETIMEDOUT 50.17.240.35:443

$ apm upgrade
Request for package information failed: connect ETIMEDOUT 50.17.187.125:443

$ apm upgrade
Request for package information failed: connect ENETUNREACH 54.243.71.103:443

$ apm upgrade
Request for package information failed: connect ENETUNREACH 50.16.214.206:443

$ apm upgrade
Request for package information failed: connect ENETUNREACH 54.243.225.13:443

$ apm upgrade
Request for package information failed: connect ETIMEDOUT 50.19.95.187:443

$ apm upgrade
Package Updates Available (3)
├── linter-pylint 2.0.1 -> 2.1.0
├── linter-ui-default 1.2.4 -> 1.6.0
└── vim-mode-plus 0.91.0 -> 0.93.0

Would you like to install these updates? (yes)

Installing [email protected] to /home/aaron/.atom/packages ✗
Request for package information failed: connect ENETUNREACH 23.23.159.159:443 (ENETUNREACH)

$ apm upgrade
Request for package information failed: connect ENETUNREACH 23.23.160.170:443

$ apm upgrade
Request for package information failed: connect ENETUNREACH 50.16.192.69:443

$ apm upgrade
Request for package information failed: connect ENETUNREACH 50.16.231.196:443

$ apm upgrade
Request for package information failed: connect ETIMEDOUT 50.16.231.196:443

$ apm upgrade
Package Updates Available (3)
├── linter-pylint 2.0.1 -> 2.1.0
├── linter-ui-default 1.2.4 -> 1.6.0
└── vim-mode-plus 0.91.0 -> 0.93.0

Would you like to install these updates? (yes)

Installing [email protected] to /home/aaron/.atom/packages ✓
Installing [email protected] to /home/aaron/.atom/packages ✓
Installing [email protected] to /home/aaron/.atom/packages ✓