Npm-check-updates: ncu -g breaks with Node binary from snap on Ubuntu

[kleinfreund]

• [x] I have read the list of known issues before filing this issue.

• [x] I have searched for similiar issues before filing this issue.

• node version: v12.1.0, v14.10.1

• npm version: 6.9.0, 6.14.8
• npm-check-updates version: 3.1.8, 9.0.1
• cross-posted bug report for node: https://github.com/nodejs/node/issues/27557
• cross-posted bug report for node distributions: https://github.com/nodesource/distributions/issues/843
• similar bug report in node: https://github.com/nodejs/node/issues/32430

---

After upgrading to Node 12 (from 11), using ncu -g produces the following error:

/home/phil/.npm-global/lib/node_modules/npm-check-updates/lib/npm-check-updates.js:387
        throw err;
        ^

Error: Expected JSON from "npm ls". This could be due to npm instability.


    at parseJson (/home/phil/.npm-global/lib/node_modules/npm-check-updates/lib/package-managers/npm.js:38:15)
    at /home/phil/.npm-global/lib/node_modules/npm-check-updates/lib/package-managers/npm.js:145:30
    at processTicksAndRejections (internal/process/task_queues.js:88:5)

Using ncu in projects works fine.

Update: This does only seem to happen when installing Node 12 via https://snapcraft.io/node:

sudo snap install node --channel=12/stable --classic

Upgrading Node to version 12 via the PPA does not break ncu -g.

Source of the issue

This issue is caused by child_process.spawn when the Node/NPM binaries are installed via Snap.

Minimal reproduction:

const { spawnSync } = require('child_process')

// Both these spawn commands will yield nothing in their stdout.
// The latter should just contain a Node version tring such as `v14.10.1`.

// npm ls --global --depth=0 --json
const npmListOutput = String(spawnSync('npm', ['ls', '--global', '--depth=0', '--json']).stdout)
console.log({ npmListOutput });

// node --version
const nodeVersionOutput = String(spawnSync('node', ['--version']).stdout)
console.log({ nodeVersionOutput });

[raineorshine]

Thanks for reporting. I am unable to reproduce on OSX (node v12.1.0). I would guess that this is not an issue with npm-check-updates.

[kleinfreund]

@raineorshine It only happens when using the Node binary distributed with Ubuntu’s snap. The binary from the PPA works fine. Do you think this should be filed with Node?

[raineorshine]

Do you think this should be filed with Node?

Yes, as node v12 uses npm v6.9.0 just like node v11. You could try to reproduce directly with npm ls --json. ncu is expecting that the results are valid JSON.

[kleinfreund]

The constructed command is

npm ls --global --depth=0 --json

and the produced JSON is valid. I’ll install the other binary on another system and see what’s different.

Update: Everything looks identical. The spawn call to npm ls in npm-check-updates produces an empty string with the broken binary. When running the command from terminal, I get the expected JSON output.

The spawn-please promise resolves with an empty string in stdout and get arguments passed to child_process.spawn are identical. That makes this a Node issue, I believe.

[kleinfreund]

This is still happening. I cannot use the snap binary for Node ever since this was initially reported. The symptoms are exactly the same now. I added the current version numbers to the original post. I think this is a problem with npm-check-updates.

[raineorshine]

Thanks! Can you try console.log('result') in the list function at /home/phil/.npm-global/lib/node_modules/npm-check-updates/lib/package-managers/npm.js:38:15 and see what you get back? That might give us a clue.

[kleinfreund]

Sure! Unfortunately, it’s just the empty string. That leads me to believe that the ls command is done in the wrong place. Logging options yields an option with prefix: null. I should note that my ~/.npmrc contains a prefix setting and that my node modules reside in ~/.npm-global (i.e. my .npmrc file has prefix=/home/phil/.npm-global). Maybe npm-check-updates doesn’t read that configuration?

Update: Using ncu -g --prefix="/home/phil/.npm-global" doesn’t change the result, unfortunately.

Update 2: npm ls --global --prefix=/home/phil/.npm-global --depth=0 --json produces the expected output in JSON format.

This also looks to be the same issue as https://github.com/nodejs/node/issues/32430. spawn produces an empty string for a command that shouldn’t produce an empty string.

Update 3: Somehow await spawn('/snap/bin/npm', ['ls', '--global', '--depth=0']) always yields the empty string.

[raineorshine]

Great, that's helpful. It would make sense that it's a prefix problem. There is some platform-specific logic for getting the npm prefix.

I'm guessing the problem is somewhere in here:
https://github.com/raineorshine/npm-check-updates/blob/master/lib/package-managers/npm.js#L197-L234

Does npm config get prefix return /home/phil/.npm-global?
Does the prefix variable in defaultPrefix get set to /home/phil/.npm-global? (presumably not)
What about npmConfig.prefix?

[kleinfreund]

Does the prefix variable in defaultPrefix get set to /home/phil/.npm-global? (presumably not)

No, it does not. It would seem that all calls to spawn using the npm command end up returning the empty string. Maybe there is an obscoure permission issue with the snap binary (i.e. /snap/bin/npm)?

What about npmConfig.prefix?

I assume you mean npmOptions? If I provide the --prefix option, it’s correct; if I don’t, it’s the empty string.

---

For what it’s worth, both spawn calls that use npm for the cmd argument produce the empty string, so I think this is where the culprit must lie.

Update: In /home/phil/.npm-global/lib/node_modules/npm-check-updates/node_modules/spawn-please/index.js:32, the callback is never called when I run ncu -g implying that it never reads any data from child.stdout which explains why spawn resolves to stdout === "".

[raineorshine]

What about npmConfig.prefix?

I assume you mean npmOptions?

No, the npmOptions variable in https://github.com/raineorshine/npm-check-updates/blob/master/lib/package-managers/npm.js.

Update: In /home/phil/.npm-global/lib/node_modules/npm-check-updates/node_modules/spawn-please/index.js:32, the callback is never called when I run ncu -g implying that it never reads any data from child.stdout which explains why spawn resolves to stdout === "".

Does it read data from child.stderr? Often stdout is empty when there is an error.

Maybe it's an issue with spawn-please which is admittedly home grown. Could you do git clone https://github.com/raineorshine/spawn-please && cd spawn-please && npm install && npm test and let me know if the tests pass?

[kleinfreund]

Does it read data from child.stderr? Often stdout is empty when there is an error.

There is nothing logged for either the stdout or stderr data callbacks.

Could you do git clone https://github.com/raineorshine/spawn-please && cd spawn-please && npm install && npm test and let me know if the tests pass?

Sure. Unfortunately, we got one failing test:

npm WARN lifecycle The node binary used for scripts is /snap/bin/node but npm is using /snap/node/2913/bin/node itself. Use the `--scripts-prepend-node-path` option to include the path for the node binary npm was executed with.

> [email protected] test /home/phil/Downloads/spawn-please
> mocha

STDOUT
STDERR


  spawn-please
    ✓ resolve on success
    ✓ reject on fail
    ✓ allow errors to be ignored with rejectOnError:false
    ✓ handle command-line arguments
    ✓ accept stdin
    ✓ allow you to specify a custom Promise
    ✓ accept options as second argument
    ✓ accept options as third argument
    1) expose stdout and stderr


  8 passing (59ms)
  1 failing

  1) spawn-please expose stdout and stderr:

      AssertionError: expected '' to equal 'STDERR'
      + expected - actual

      +STDERR

      at test/spec.js:78:36
      at processTicksAndRejections (internal/process/task_queues.js:97:5)



npm ERR! Test failed.  See above for more details.

The start of the output looks highly somewhat suspicious to me.

[raineorshine]

Does this make any difference?

npm config set scripts-prepend-node-path true

https://github.com/nwjs/npm-installer/issues/73

[kleinfreund]

Setting scripts-prepend-node-path to true makes the tests for spawn-please pass and the output looks clean now. As for ncu -g, it didn’t change anything as far as I can tell.

[raineorshine]

Hmmm. It's possible that scripts-prepend-node-path is not being read from the config correctly.

What happens if you manually add --scripts-prepend-node-path to the npm arguments in spawnNpm?

[kleinfreund]

What happens if you manually add --scripts-prepend-node-path to the npm arguments in spawnNpm?

No change. The result variable still contains the empty string.

[raineorshine]

We know that npm ls --global --prefix=/home/phil/.npm-global --depth=0 --json works correctly when you run it in your shell.

Can you do console.log(`${cmd} ${fullArgs.join(' ')}`) in spawnNpm to confirm that ncu is using the same npm command?

We seemed to have fixed spawn-please, so maybe the arguments are not correct.

[kleinfreund]

Yes, of course. In fact, it’s how I constructed the command in the first place. The output is:

npm ls --global --depth=0 --json --scripts-prepend-node-path

I also copied it and verified that exactly this string produces the correct output.

[raineorshine]

What if you add --prefix=/home/phil/.npm-global directly to the spawnNpm arguments?

Sorry if you already tried this.

[kleinfreund]

Also nothing, sadly. And no worries, this is unexpectedly strange to debug. Setting it there or directly when running ncu -g --prefix /home/phil/.npm-global has the same result (and it’s passed along correctly!).

[raineorshine]

Okay, last shot: ncu -g --prefix "/home/phil/.npm-global" --cwd "/home/phil/.npm-global".

Otherwise, I'm stumped!

[kleinfreund]

It pains me to say this, but no dice. It’s the same result once more. Thank you for the investigation in any case.

[raineorshine]

Sure, thanks for your cooperation, and sorry we couldn't get to the bottom of it!

If I were troubleshooting further I might make a test script that tries to get npm ls --global --depth=0 --json --prefix=/home/phil/.npm-global returning the correct results with a child_process. Maybe there is a different method or argument you can pass to the child_process API that would solve it.

[kleinfreund]

Even this program logs true. This must be an issue with the snap binaries after all.

const npm = require('child_process').spawnSync('npm', ['ls', '--global', '--depth=0', '--json', '--prefix=/home/phil/.npm-global'])
console.log(String(npm.stdout) === '');

[kleinfreund]

I’m going to close this because it’s really not a problem with npm-check-updates. Nothing to be done here. Thank you again for your help.