Cli: npm publish: error network timeout

[hismightiness]

What / Why

I've been trying to publish the latest release of my npm package and it kept reporting the error that the problem was a timeout with the registry URL. I tried many things, but finally ran in the issue in the original repo below.

https://github.com/npm/npm/issues/129

The same fix where we manually specify the non-secure URL for the registry appears to actually publish the package, but the same network timeout error is still reported.

When

• cd to the folder where the package source is found
• Type npm publish and press
• Wait...
• A network timeout error occurs in the node-fetch-npm package
• I explicitly updated npm, node, and node-fetch-npm packages
• Tried it all over again (no change)
• Did lots of Googling with no relevant results. Finally found the post mentioned above.
• Ran npm publish --registry http://registry.npmjs.org/ and the same timeout error was reported, but the package publish e-mail was received quickly.

Package Page: https://www.npmjs.com/package/generator-upendodnn

I can provide the log file details if you'd like, but here are the relevant details at the end:

Without the Registry Specified:

18 verbose type request-timeout
19 verbose stack FetchError: network timeout at: https://registry.npmjs.org/generator-upendodnn
19 verbose stack     at Timeout.setTimeout [as _onTimeout] (C:\Users\stroh\AppData\Roaming\npm\node_modules\npm\node_modules\node-fetch-npm\src\index.js:61:18)
19 verbose stack     at ontimeout (timers.js:436:11)
19 verbose stack     at tryOnTimeout (timers.js:300:5)
19 verbose stack     at listOnTimeout (timers.js:263:5)
19 verbose stack     at Timer.processTimers (timers.js:223:10)
20 verbose cwd c:\Work\Generator-UpendoDNN\generator-upendodnn
21 verbose Windows_NT 10.0.17763
22 verbose argv "C:\\Program Files\\nodejs\\node.exe" "C:\\Users\\stroh\\AppData\\Roaming\\npm\\node_modules\\npm\\bin\\npm-cli.js" "publish"
23 verbose node v10.15.0
24 verbose npm  v6.14.2
25 error network timeout at: https://registry.npmjs.org/generator-upendodnn
26 verbose exit [ 1, true ]

With the Registry Specified:

18 verbose type request-timeout
19 verbose stack FetchError: network timeout at: http://registry.npmjs.org/generator-upendodnn
19 verbose stack     at Timeout.setTimeout [as _onTimeout] (C:\Users\stroh\AppData\Roaming\npm\node_modules\npm\node_modules\node-fetch-npm\src\index.js:61:18)
19 verbose stack     at ontimeout (timers.js:436:11)
19 verbose stack     at tryOnTimeout (timers.js:300:5)
19 verbose stack     at listOnTimeout (timers.js:263:5)
19 verbose stack     at Timer.processTimers (timers.js:223:10)
20 verbose cwd c:\Work\Generator-UpendoDNN\generator-upendodnn
21 verbose Windows_NT 10.0.17763
22 verbose argv "C:\\Program Files\\nodejs\\node.exe" "C:\\Users\\stroh\\AppData\\Roaming\\npm\\node_modules\\npm\\bin\\npm-cli.js" "publish" "--registry" "http://registry.npmjs.org/"
23 verbose node v10.15.0
24 verbose npm  v6.14.2
25 error network timeout at: http://registry.npmjs.org/generator-upendodnn
26 verbose exit [ 1, true ]

[mykola-mokhnach]

I experience the same error, but only with some particular packages, for example :

12 notice 📦  [email protected]
13 notice === Tarball Contents ===
14 notice 10.3kB  LICENSE
14 notice 552.0kB apks/appium-uiautomator2-server-debug-androidTest.apk
14 notice 2.2MB   apks/appium-uiautomator2-server-v4.6.3.apk
14 notice 362B    index.js
14 notice 1.7kB   package.json
14 notice 1.7kB   README.md
15 notice === Tarball Details ===
16 notice name:          appium-uiautomator2-server
16 notice version:       4.6.3
16 notice package size:  2.8 MB
16 notice unpacked size: 2.8 MB
16 notice shasum:        d202b85c74985dfdb2e9dfb50bc9bd27667bee86
16 notice integrity:     sha512-ZjFFBM2Ud9pOx[...]xlJflS9H80bcA==
16 notice total files:   6
17 notice
18 verbose type request-timeout
19 verbose stack FetchError: network timeout at: https://registry.npmjs.org/appium-uiautomator2-server
19 verbose stack     at Timeout._onTimeout (/usr/local/lib/node_modules/npm/node_modules/node-fetch-npm/src/index.js:61:18)
19 verbose stack     at listOnTimeout (internal/timers.js:549:17)
19 verbose stack     at processTimers (internal/timers.js:492:7)
20 verbose cwd /Users/elf/code/appium-uiautomator2-server
21 verbose Darwin 19.4.0
22 verbose argv "/usr/local/Cellar/node/13.12.0/bin/node" "/usr/local/bin/npm" "publish"
23 verbose node v13.12.0
24 verbose npm  v6.14.4
25 error network timeout at: https://registry.npmjs.org/appium-uiautomator2-server
26 verbose exit [ 1, true ]

Is there any good workaround, where, for example the actual timeout could be customized?

I use no proxy and the URL above is opened pretty fine if curl or a browser is used.

[tobiu]

Adding some weight here, this is getting worse and worse for me with the https://github.com/neomjs/neo repo.

passing a registry url does not make a difference for me.

using curl on the registry file is almost done instantly.

at the moment i need to call publish at least 30 times, until i get an email that it worked. the call which worked is not distinguishable on the terminal: throws the same error.

npm ERR! network timeout at: http://registry.npmjs.org/neo.mjs

npm ERR! A complete log of this run can be found in:
npm ERR! /Users/tobiasuhlig/.npm/_logs/2020-05-24T01_12_40_101Z-debug.log

it is definitely related to the filesize => the more total releases are out there, the higher is the chance to break.

[tobiu]

It is still a nightmare. In average 30+ attempts now to get just one npm publish through.

if it happens to work, there is still a "timeout fail" error, but you get an email later on that it did come through.

very hard to automate this.

an official comment is still missing here, which is not good.

[Jikstra]

Having the same problem over at https://github.com/deltachat/deltachat-node.git . Very annoying.

[mieszko4]

If I recall correctly someone had a problem with it when using mobile internet.
Is that maybe a case for you?
~I am having the same issue, but I want to check with no-mobile to confirm.~
It worked for me on mobile internet after 5th attempt. I switched from 4G to 3G though - not sure if that mattered.

[tobiu]

Not using mobile internet.

Provider MNet (Germany), I think it is 60MBit/s glass fibre.

Extremely fast for everything I do.

[Jikstra]

Having 100mbit here, still same problem.

[mieszko4]

I am again having problem with this :(
I will run repeat script until succeed like:

until npm publish; do echo "Try again"; sleep 2; done

[mieszko4]

@isaacs Are there some logs on the npmjs.org server regarding the timeouts? Maybe they reveal more regarding the issue.
I am trying to publish http://npmjs.org/react-native-zoom-us

[mieszko4]

Notes

I've played more with this and here are my findings:

1. I am trying to upload a large file ~100mb from Europe to the upload server in America/California (dig registry.npmjs.org)
   When I analyze the traffic it looks like the upload goes to about 100mb then pauses then goes to another 100mb then pauses etc. until timeout happens (~150s).
2. When I analyzed network traffic in wireshark I've noticed a lot of TCP Out-of-Order, TCP Fast Retransmission
3. When sshed to a AWS EC2 instance I have in us-east-2 region and run npm publish, the upload succeeded immediately

Conclusion

So my conclusion is that the issue is not slow internet but unstable internet which causes the entire file to reupload each time some error happens on TCP level.
I am not sure what the fix should be here because I do not know the internals of the server and the implementation of the client. To me it looks like some TCP recovery mechanism is missing. Some suggestions:

• on the upload server (not sure if it is cloudfare?), e.g turning on TCP SACK perhaps (https://tools.ietf.org/html/rfc2018)
• updating lib for upload mechanism?

Workaround

Run npm publish from a remote instance located in America/California so that the network connection is more stable. As an idea you could set up a github action on tag push (my example https://github.com/mieszko4/react-native-zoom-us/blob/master/.github/workflows/npm-publish.yml)

[tobiu]

My current workaround: I am using a GitHub Action to call npm publish whenever i add a release in GitHub.

This resolved the timeout issue for me. GH has obviously a faster connection and data does not need to travel from Europe around the world. Might be a temporary fix until the registry file gets too big again (200 releases so far). At this point it will be time for a new npm package anyway (version2 i guess).

Even if your code for your npm package is not inside a github project, you could still do it this way. Create a trigger event, get the latest version of your code, publish it.

https://github.com/neomjs/neo/blob/dev/.github/workflows/npm-publish.yml