Termux-packages: npm install commands fail on Android 8 - os.cpus() returns undefined

[shannon]

I don't know if this a node issue or a termux issue but I figured I would start here.

If I install nodejs via pkg install nodejs whenever I run npm commands I get an error

$ npm i -g http-server
npm ERR!  Cannot read property 'length' of undefined

npm ERR!  A complete log of this run can be found in:
npm ERR!       /data/data/com.termux/files/home/.npm/_logs/2017-11-15T15_21_06_474Z-debug.log

After reviewing the logs I tracked the error down to this line in a npm dependency.

It seems that os.cpus() returns undefined. I confirmed this using the node command prompt.

> var os = require('os')
undefined
> os.cpus()
undefined

I also tried nodejs-current but no luck. Could this be an issue with termux?

[shannon]

I should also note, other os methods seem to work ok.

```bash

os.arch()
'arm64'
os.release()
'4.4.56-g594d847d09a1'
os.platform()
'android'
os.type()
'Linux'

[Quasic]

Well, the wrapper function is rather simple, so I'm not sure if this will help, but you can try this test script:
javascript const cpuValues = new Float64Array(6 * process.binding("util").pushValToArrayMax); function f(){console.log(arguments);} console.log(process.binding("os").getCPUs(f,cpuValues,[])); console.log(cpuValues);
@shannon, I edited this to log the [], as well, so now this should log an Arguments array of cpus, then the [], then an array of numbers, if everything's working. If it lists undefined instead of one or more of these, it might narrow down the problem a bit.

[fornwall]

@shannon Are you running the latest version of the nodejs package (node --version == v8.9.1)?

I tested os.cpus() on an aarch64 device and it worked, but this was with a 3.10.84-perf-g99119bc kernel. Could it be kernel specific (or perhaps depending on SELinux rules)?

[shannon]

@Quasic I tried this and it outputs the following:

(I truncated all the 0s)

$ node test.js
undefined
Float64Array [
  0,
  0,
  0,
...
]

@fornwall yes I tried version 8.9.1 and 9.2.0. Both result in the same issue.

I just got this phone last week and I haven't done anything in termux yet so I don't think it was anything I did. I even uninstalled an reinstalled termux to see if it would resolve it.

I did notice that my os.arch() says "arm64" and not "aarch64" but I assume this is ok.

[Quasic]

Just for the record, it's working in mine, which is

os.arch()
'arm'
os.release()
'3.10.49-gecffe94'

$ node --version
v8.9.1

[shannon]

So I tried it on my wife's Pixel XL (not 2) and I get the same error.

os.arch()
"arm64"
os.release()
"3.18.52-g99dda0323132"

$ node --version
v8.9.1

[Quasic]

@shannon
Well, I doubt it's setting the timing array, with no nonzero values.
It also appears to be ignoring the callback?

You could see if "callback" is logged (should be on the first line) and see if the 0s are all replaced by -1 when running this:
javascript const L=6 * process.binding("util").pushValToArrayMax; const cpuValues = new Float64Array(L); for(var i=0;i<L;i++)cpuValues[i]=-1; function f(){console.log("callback");console.log(arguments);} console.log(process.binding("os").getCPUs(f,cpuValues,[])); console.log(cpuValues);
I think the native function is basically acting empty.

[shannon]

@Quasic This script outputs all -1 now so it does seem to be calling the callback.

I found a similar bug reported against freebsd here:

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=221762

It seems that if the something doesn't work in libuv it might get be causing it to returned undefined. I don't think the patch provided in that bug applies here but it could give us a lead. I don't really know how to go about testing this functionality in libuv though.

[brianppoole]

I seem to be running into the same issue on my Pixel 2 running Node v.8.9.1.

$ npm i -g http-server
npm ERR! Cannot read property 'length' of undefined

npm ERR! A complete log of this run can be found in:
npm ERR!     /data/data/com.termux/files/home/.npm/_logs/2017-11-16T00_08_35_087Z-debug.log

$ node
> var os = require("os")
undefined
> os.arch()
'arm64'
> os.release()
'4.4.56-g594d847d09a1'

Anything I can do to help diagnose?

[Quasic]

@shannon, the -1 test was to verify that it wasn't changing the Float64Array. The callback should log the word "callback" if it is called.

I found that the Windows variant apparently used to return -1 ?? before https://github.com/nodejs/node-v0.x-archive/commit/9557020

[shannon]

@Quasic oh sorry I misread what you wrote. "callback" is NOT output so you are correct.

node test.js
undefined
Float64Array [
  -1,
  -1,
  -1,
  -1,
...

[Quasic]

The patch attached to the last post of the bugzilla report looks promising...

[Quasic]

https://github.com/nodejs/node/blob/c39caa997c751473d0c8f50af8c6b14bcd389fa0/deps/v8/src/base/cpu.cc seems to test for arm64...

[Quasic]

@shannon, @brianppoole Is this on Android Nougat or Oreo? https://github.com/nodejs/node/blob/master/deps/uv/src/unix/linux-core.c uses /proc/stat to retrieve the cpu info, which means termux/termux-app#299 could apply here.

[shannon]

@Quasic, yea this is on Oreo 8.0.0. That looks like the problem alright :-/

[Quasic]

libuv/libuv#1459

[shannon]

thanks @Quasic

I've opened an issue here https://github.com/npm/npm/issues/19170. It would be good if npm worked without relying on this functionality or gracefully handled the lack of it.

I was able to workaround the issue myself by changing /data/data/com.termux/files/usr/lib/node_modules/npm/node_modules/worker-farm/lib/farm.js to use a hardcoded 4 cpus. This is not really ideal though for obvious reasons.

[Quasic]

I made a PR that falls back to 1, but hopefully someone who knows the code will know a better way.

[Quasic]

npm using the current node-worker-farm won't fail, but it will disable concurrent workers. I started patching libuv, which requires more /proc/stat info be mapped to Android sources than just virtual cpu number. If it works, maybe we could use proot or something to generate /proc/stat? I also have perhaps a less ambitious option, in case this doesn't work, suggested at node-worker-farm.

[SDRausty]

https://github.com/npm/npm/issues/19239 This is an error with npm itself. Please report this error... #19239

[Quasic]

It was reported in npm/npm#19170

[F1LT3R]

Thanks @shannon, I did what you did. But I changed: /data/data/com.termux/files/usr/lib/node_modules/npm/node_modules/worker-farm/lib/farm.js to use 4 maxConcurrentWorkers in DEFAULT_OPTIONS. Works perfectly on the Pixel 2 XL.

[gund]

Same error on Pixel 2 device and both stable and current versions of NodeJs.

Patching farm.js works for now but obviously is not a good solution...

[F0RIS]

Why npm still is not fixed!!?

[fornwall]

The nodejs package has been updated to the current 10.10.0 version of node. Does this work better now?

[MaxPelly]

The nodejs package has been updated to the current 10.10.0 version of node. Does this work better now?

No, still returning undefined

[xeffyr]

This error is related to unreadable /proc/stat file.

If I copy this file from another device (PC) and bind it with proot, then os.cpus() will work:

1. Test script (./test.js):
   ``` .js
   var os = require("os")
   console.log(os.cpus())

2. Dummy file (./stat) for /proc/stat:

cpu 5115690 2798653 1880760 7948594 13293 227304 70323 0 0 0
cpu0 2441336 1338502 1065622 6886762 11873 173371 50618 0 0 0
cpu1 2674353 1460151 815138 1061832 1420 53933 19705 0 0 0
intr 325594253 9 40544 0 0 0 0 0 0 1 26465 0 0 24398555 0 0 0 1484163 0 0 0 0 0 0 149 0 0 666186 3551761 21 722 61752469 5569088 8993 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
ctxt 801588897
btime 1536941305
processes 2420179
procs_running 1
procs_blocked 0
softirq 130904467 1473985 60414110 76040 726869 3416486 5 3515545 29874514 145808 31261105

3. When using original /proc/stat (this will fail on Android 8):
``` .bash
$ node test.js
undefined

1. Binding custom file to /proc/stat with proot (this will work):
   .bash $ proot -b ./stat:/proc/stat node test.js [ { model: 'unknown', speed: 442, times: { user: 244133600, nice: 133850200, sys: 106562200, idle: 688676200, irq: 17337100 } }, { model: 'unknown', speed: 442, times: { user: 267435300, nice: 146015100, sys: 81513800, idle: 106183200, irq: 5393300 } } ]

[daveball]

This problem still exist but I think if you use yarn instead of npm and clone node-red from github the it work, just trying this now do will update you all once I've confirmed

[imkiva]

As is said in another issue:

Starting in Android Nougat, apps are no longer allowed to see running processes.

But if you set your API level to 22, you will have the permission to do it.

[xeffyr]

@imkiva SELinux permission denial. It can't be fixed.

set your API level to 22

No thanks, I don't want to downgrade my devices to API 22.

[imkiva]

SELinux permission denial. It can't be fixed.

@xeffyr Try before say.
Another project just fixed this.

I don't want to downgrade my devices to API 22.

It's up to you. No sacrifice, no gain.

[xeffyr]

Okay, I will try that.
But I don't think that file /proc/stat will suddenly become readable otherwise Android has a serious flaw in SELinux implementation.

[imkiva]

@xeffyr
I just know that the removal of /proc/stat was intentional. I didn't try /proc/stat but some other files (such as /proc/self).
If apps really can't access /proc/stat, maybe it's really a problem with SELinux.

[xeffyr]

As is said in another issue:
Starting in Android Nougat, apps are no longer allowed to see running processes.
But if you set your API level to 22, you will have the permission to do it.

Okay, I will try that.

Just built nodejs with setting TERMUX_PKG_API_LEVEL to 22. Nothing changed (I wrote above why) !
nodejs_11.6.0_aarch64.deb.gz

os.cpus() returns undefined because file /proc/stat is inaccessible. It also don't have any relation to "running processes" so as I wrote in https://github.com/termux/termux-packages/issues/1798#issuecomment-453567634, your "solution" does not fix anything.

[xeffyr]

Closing since npm no longer produces mentioned error. os.cpus() still returns undefined, but not so much can be done with that.