Patchwork: 3.18-1 beta fails on aarch64 source build

[sunjam]

I'm having non-stop troubles starting the latest Patchwork release while attempting to build it from source using #1324 neon-arm64 branch. Any guidance appreciated. Here is what I'm seeing.

$ npm start

[email protected] start /home/rock/patchwork
electron index.js

/home/rock/patchwork/node_modules/electron/index.js:14
throw new Error('Electron failed to install correctly, please delete node_modules/electron and try installing again')
^

Error: Electron failed to install correctly, please delete node_modules/electron and try installing again
at getElectronPath (/home/rock/patchwork/node_modules/electron/index.js:14:11)
at Object. (/home/rock/patchwork/node_modules/electron/index.js:18:18)
at Module._compile (module.js:569:30)
at Object.Module._extensions..js (module.js:580:10)
at Module.load (module.js:503:32)
at tryModuleLoad (module.js:466:12)
at Function.Module._load (module.js:458:3)
at Module.require (module.js:513:17)
at require (internal/module.js:11:18)
at Object. (/home/rock/patchwork/node_modules/electron/cli.js:3:16)
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] start: electron index.js
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] start script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

[black-puppydog]

Hi there :)

yeah, figuring out how to build with neon for arm was actually the main point of the beta. I already looked into it for a bit. It seems to be mostly a matter of build environment. See https://github.com/ssbc/patchwork/issues/1324
Thanks for looking into this in any case!

If you just want to run patchwork, then running off the last release (or master before the neon commits) is of course the easiest. But I'm sure you know that. :)

[sunjam]

Thanks, I've switched back to master and am updating everything related to
npm. Unsure of whether 'latest' or current 'stable' are supported.

I'm currently running 'latest'

On Fri, Nov 13, 2020, 12:12 PM Daan Wynen [e-mail] wrote:

Hi there :)

yeah, figuring out how to build with neon for arm was actually the main
point of the beta. I already looked into it for a bit. It seems to be
mostly a matter of build environment. See #1324
https://github.com/ssbc/patchwork/issues/1324
Thanks for looking into this in any case!

If you just want to run patchwork, then running off the last release (or
master before the neon commits) is of course the easiest. But I'm sure
you know that. :)

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/ssbc/patchwork/issues/1326#issuecomment-727008059,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AANUKZUNZ2A44YA5RBUCZTDSPWHMHANCNFSM4TU7DCLA
.

[sunjam]

$ npm start

[email protected] start /home/rock/patchwork
electron index.js

App threw an error during load
Error: No native build was found for platform=linux arch=arm64 runtime=electron abi=82 uv=1 armv=8 libc=glibc node=12.16.3 electron=10.1.5
loaded from: /home/rock/patchwork/node_modules/ssb-keys

at Function.load.path (/home/rock/patchwork/node_modules/neon-load-or-build/index.js:56:9)
at load (/home/rock/patchwork/node_modules/neon-load-or-build/index.js:21:30)
at Object.<anonymous> (/home/rock/patchwork/node_modules/ssb-keys/lib/index.js:1:47)
at Module._compile (internal/modules/cjs/loader.js:1145:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:1166:10)
at Module.load (internal/modules/cjs/loader.js:981:32)
at Module._load (internal/modules/cjs/loader.js:881:14)
at Function.Module._load (electron/js2c/asar.js:769:28)
at Module.require (internal/modules/cjs/loader.js:1023:19)
at require (internal/modules/cjs/helpers.js:77:18)

Error: No native build was found for platform=linux arch=arm64 runtime=electron abi=82 uv=1 armv=8 libc=glibc node=12.16.3 electron=10.1.5
loaded from: /home/rock/patchwork/node_modules/ssb-keys

at Function.load.path (/home/rock/patchwork/node_modules/neon-load-or-build/index.js:56:9)
at load (/home/rock/patchwork/node_modules/neon-load-or-build/index.js:21:30)
at Object.<anonymous> (/home/rock/patchwork/node_modules/ssb-keys/lib/index.js:1:47)
at Module._compile (internal/modules/cjs/loader.js:1145:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:1166:10)
at Module.load (internal/modules/cjs/loader.js:981:32)
at Module._load (internal/modules/cjs/loader.js:881:14)
at Function.Module._load (electron/js2c/asar.js:769:28)
at Module.require (internal/modules/cjs/loader.js:1023:19)
at require (internal/modules/cjs/helpers.js:77:18)

[black-puppydog]

Yes, the "no native build" error message is from the neon bindings for rust.

Just to be clear: currently, master is the same commit from which the last beta was built and will not build on arm64.
I just pushed a tag pre-rust that marks the last "good" commit.

[sunjam]

Thanks, I'd just like to clarify how this process works.
git checkout -b pre-rust

Unsure on what to do next to get Patchwork up and running locally. Still wrapping my head around this.

[black-puppydog]

@sunjam no, if you do checkout -b this will create a branch called pre-rust, but keep that branch at the same commit as where you are currently. To switch to the tag, simply do git checkout pre-rust (without "-b"). That will print some warnings about not being on a branch any longer. So if you then want to create a branch at the same commit, you can follow the advice in the warning and do git switch -c rust-free to create the rust-free branch. But that isn't necessary to build the application.

As for building... since you've been experimenting a bunch and have god-knows-which state in the build cache, I personally would simply rm -r node_modules to start from a blank slate. Theoretically, running npm ci should be enough then (and then npm start to launch of course) but since you're on arm... If npm ci fails, better run npm i which will take longer but should have a better chance of working.

I'll try this on my arm machine without the rust bindings... let's see. :)

[black-puppydog]

Okay I just confirmed that the above works for me on the r400

[sunjam]

I haven't updated npm in about a year and seems I'm on 8.0.0 when modern stable is 15.x so the update process is a total nightmare. :( Still struggling

[black-puppydog]

@sunjam I think if you remove node_modules and then rebuild with modern npm, it should not really be an upgrade process?
Or do you mean you didn't upgrade patchwork in a year? I don't think we changed thaaaat much in the last year, so the upgrade should mostly be maintaenance stuff....

[sunjam]

0 info it worked if it ends with ok
1 verbose cli [ '/home/rock/.nvm/versions/node/v8.0.0/bin/node',
1 verbose cli   '/home/rock/.nvm/versions/node/v8.0.0/bin/npm',
1 verbose cli   'start' ]
2 info using [email protected]
3 info using [email protected]
4 verbose config Skipping project config: /home/rock/.npmrc. (matches userconfig)
5 verbose stack Error: missing script: start
5 verbose stack     at run (/home/rock/.nvm/versions/node/v8.0.0/lib/node_modules/npm/lib/run-script.js:155:19)
5 verbose stack     at /home/rock/.nvm/versions/node/v8.0.0/lib/node_modules/npm/lib/run-script.js:63:5
5 verbose stack     at /home/rock/.nvm/versions/node/v8.0.0/lib/node_modules/npm/node_modules/read-package-json/read-json.js:116:5
5 verbose stack     at /home/rock/.nvm/versions/node/v8.0.0/lib/node_modules/npm/node_modules/read-package-json/read-json.js:436:5
5 verbose stack     at checkBinReferences_ (/home/rock/.nvm/versions/node/v8.0.0/lib/node_modules/npm/node_modules/read-package-json/read-json.js:391:45)
5 verbose stack     at final (/home/rock/.nvm/versions/node/v8.0.0/lib/node_modules/npm/node_modules/read-package-json/read-json.js:434:3)
5 verbose stack     at then (/home/rock/.nvm/versions/node/v8.0.0/lib/node_modules/npm/node_modules/read-package-json/read-json.js:161:5)
5 verbose stack     at ReadFileContext.<anonymous> (/home/rock/.nvm/versions/node/v8.0.0/lib/node_modules/npm/node_modules/read-package-json/read-json.js:350:20)
5 verbose stack     at ReadFileContext.callback (/home/rock/.nvm/versions/node/v8.0.0/lib/node_modules/npm/node_modules/graceful-fs/graceful-fs.js:123:16)
5 verbose stack     at FSReqWrap.readFileAfterOpen [as oncomplete] (fs.js:406:13)
6 verbose cwd /home/rock
7 verbose Linux 5.9.6-1-MANJARO-ARM
8 verbose argv "/home/rock/.nvm/versions/node/v8.0.0/bin/node" "/home/rock/.nvm/versions/node/v8.0.0/bin/npm" "start"
9 verbose node v8.0.0
10 verbose npm  v6.14.8
11 error missing script: start
12 verbose exit [ 1, true ]

Stuck on nvm 8.0.0 even though I try to switch to current stable

$ nvm list
         v6.1.0
        v6.10.0
->       v8.0.0
        v15.1.0
        v15.2.0
         system
default -> 8.0 (-> v8.0.0)
node -> stable (-> v15.2.0) (default)
stable -> 15.2 (-> v15.2.0) (default)
iojs -> N/A (default)
unstable -> N/A (default)
lts/* -> lts/fermium (-> N/A)
lts/argon -> v4.9.1 (-> N/A)
lts/boron -> v6.17.1 (-> N/A)
lts/carbon -> v8.17.0 (-> N/A)
lts/dubnium -> v10.23.0 (-> N/A)
lts/erbium -> v12.19.0 (-> N/A)
lts/fermium -> v14.15.0 (-> N/A)

$ nvm use stable
Now using node v15.2.0 (npm v6.14.8)
[rock@rock ~]$ nvm list
         v6.1.0
        v6.10.0
         v8.0.0
        v15.1.0
->      v15.2.0
         system
default -> 8.0 (-> v8.0.0)
node -> stable (-> v15.2.0) (default)
stable -> 15.2 (-> v15.2.0) (default)
iojs -> N/A (default)
unstable -> N/A (default)
lts/* -> lts/fermium (-> N/A)
lts/argon -> v4.9.1 (-> N/A)
lts/boron -> v6.17.1 (-> N/A)
lts/carbon -> v8.17.0 (-> N/A)
lts/dubnium -> v10.23.0 (-> N/A)
lts/erbium -> v12.19.0 (-> N/A)
lts/fermium -> v14.15.0 (-> N/A)

$ nvm install --latest-npm
No .nvmrc file found
v15.2.0 is already installed.
Now using node v15.2.0 (npm v6.14.8)
Attempting to upgrade to the latest working version of npm...
* Installing latest `npm`; if this does not work on your node version, please report a bug!
/home/rock/.nvm/versions/node/v15.2.0/bin/npm -> /home/rock/.nvm/versions/node/v15.2.0/lib/node_modules/npm/bin/npm-cli.js
/home/rock/.nvm/versions/node/v15.2.0/bin/npx -> /home/rock/.nvm/versions/node/v15.2.0/lib/node_modules/npm/bin/npx-cli.js
+ [email protected]
updated 1 package in 21.603s
* npm upgraded to: v6.14.8
nvm_ensure_default_set: a version is required

[black-puppydog]

have you tried simply nvm install-latest-npm?

[sunjam]

have you tried simply nvm install-latest-npm?

Yeah, see bottom of comment above. I just feel like I'm running in circles on this. Unsure on how to rebuild this.

$ nvm run stable patchwork
Running node v15.2.0 (npm v6.14.8)
Error: Cannot find module 'electron'
Require stack:
- /home/rock/patchwork/index.js
    at Function.Module._resolveFilename (node:internal/modules/cjs/loader:919:15)
    at Function.Module._load (node:internal/modules/cjs/loader:763:27)
    at Module.require (node:internal/modules/cjs/loader:991:19)
    at require (node:internal/modules/cjs/helpers:92:18)
    at Object.<anonymous> (/home/rock/patchwork/index.js:6:18)
    at Module._compile (node:internal/modules/cjs/loader:1102:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1131:10)
    at Module.load (node:internal/modules/cjs/loader:967:32)
    at Function.Module._load (node:internal/modules/cjs/loader:807:14)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:76:12) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [ '/home/rock/patchwork/index.js' ]
}

$ npm start
npm ERR! missing script: start

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/rock/.npm/_logs/2020-11-15T16_47_53_344Z-debug.log
[rock@rock patchwork]$ npm ci
added 0 packages in 0.085s
[rock@rock patchwork]$ npm install -d
npm info it worked if it ends with ok
npm info using [email protected]
npm info using [email protected]
npm info lifecycle @~preinstall: @
npm timing stage:loadCurrentTree Completed in 14ms
npm timing stage:loadIdealTree:cloneCurrentTree Completed in 1ms
npm timing stage:loadIdealTree:loadShrinkwrap Completed in 4ms
npm timing stage:loadIdealTree:loadAllDepsIntoIdealTree Completed in 5ms
npm timing stage:loadIdealTree Completed in 13ms
npm timing stage:generateActionsToTake Completed in 6ms
npm timing stage:executeActions Completed in 93ms
npm timing stage:rollbackFailedOptional Completed in 0ms
npm info linkStuff !invalid#1
npm info lifecycle undefined~install: undefined
npm info lifecycle undefined~postinstall: undefined
npm info lifecycle undefined~prepublish: undefined
npm info lifecycle undefined~prepare: undefined
npm timing stage:runTopLevelLifecycles Completed in 224ms
npm info lifecycle undefined~preshrinkwrap: undefined
npm info lifecycle !invalid#2~shrinkwrap: !invalid#2
npm info lifecycle !invalid#2~postshrinkwrap: !invalid#2
npm WARN rock No description
npm WARN rock No repository field.
npm WARN rock No license field.

npm timing audit submit Completed in 899ms
npm http fetch POST 200 https://registry.npmjs.org/-/npm/v1/security/audits/quick 904ms
npm timing audit body Completed in 9ms
up to date in 1.046s
found 0 vulnerabilities

npm timing npm Completed in 1821ms
npm info ok 

[black-puppydog]

hmmm... after re-reading the nvm help strings I'm left unsure whether that's any different from what you did. there seem to be two distinct commands...

nvm install --latest-npm    After installing, attempt to upgrade to the latest working npm on the given node version

...

nvm install-latest-npm    Attempt to upgrade to the latest working `npm` on the current node version

edit: in any case, just to make sure, what does npm --version give you with the current setup?

[sunjam]

edit: in any case, just to make sure, what does npm --version give you with the current setup?

$ npm --version
6.14.8

@black-puppydog what is the best method to "rebuild with modern npm"? Thanks for the assistance!

[black-puppydog]

So, on x64 (!) I get these here:

daan@flunderland ~/code/patchwork 🦆3» nvm install stable                                                                                                                                                     17:56
Downloading and installing node v15.2.0...
Downloading https://nodejs.org/dist/v15.2.0/node-v15.2.0-linux-x64.tar.xz...
############################################################################################################################################################################################################# 100.0%
Computing checksum with sha256sum
Checksums matched!
Now using node v15.2.0 (npm v7.0.8)
daan@flunderland ~/code/patchwork 🦇» npm --version                                                                                                                                                           17:57
7.0.8
daan@flunderland ~/code/patchwork 🦇» node --version                                                                                                                                                          17:57
v15.2.0

and then rm -rf node_modules && npm ci works fine.

HOWEVER: when I then launch patchwork it opens a blank window and then prints a bunch of ABI mismatch errors.
So I think I'll re-try the whole thing with nvm use --lts, and in parallel I'll also check with the raspi. :)

EDIT: to clarify, this is on the pre-rust tag's commit.

[black-puppydog]

Okay, unsurprisingly the x64 was "a bit" faster. and runs fine after nvm use --lts && npm ci && npm start -- --no-sandbox

daan@flunderland ~/code/patchwork 🦇» npm --version                                                                                                                                                           
6.14.8
daan@flunderland ~/code/patchwork 🦇» node --version                                                                                                                                                         
v14.15.0

[black-puppydog]

Confirmed to be working on the raspi. So, could you try with nvm use --lts?

[sunjam]

nvm install lts/* --reinstall-packages-from=default --latest-npm
  562  nvm uninstall 8.0.0
  563  nvm list
  564  nvm alias default lts/*
  565  nvm list
  566  nvm uninstall v8.0.0
  567  nvm uninstall 8.0.0
  568  nvm uninstall 8.0
  569  nvm uninstall 6.1.0
  570  nvm uninstall 6.10.0
  571  nvm uninstall 15.1.0
  572  npm start
  573  npm use --lts
  574  npm use lts/*
  575  npm use lts
  576  npm ci
  577  npm start -- --no-sandbox
  578  ls
  579  npm install 0d
  580  npm install -d
  581  cd ..
  582  rm -rf patchwork/
  583  nvm list
  584  nvm uninstall 8.0.0
  585  nvm uninstall 8
  586  nvm list
  587   history
  588  git clone https://github.com/ssbc/patchwork.git
  589   history | grep git
  590  cd patchwork/
  591  git checkout pre-rust
  592 rm -r node_modules
  593  npm ci

Back to same kinds of errors as when I started.

Error: No native build was found for platform=linux arch=arm64 runtime=node abi=83 uv=1 armv=8 libc=glibc node=14.15.0
    loaded from: /home/rock/patchwork/node_modules/ssb-keys

    at Function.load.path (/home/rock/patchwork/node_modules/neon-load-or-build/index.js:56:9)
    at load (/home/rock/patchwork/node_modules/neon-load-or-build/index.js:21:30)
    at Object.<anonymous> (/home/rock/patchwork/node_modules/neon-load-or-build/build-test.js:19:19)
    at Module._compile (internal/modules/cjs/loader.js:1063:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1092:10)
    at Module.load (internal/modules/cjs/loader.js:928:32)
    at Function.Module._load (internal/modules/cjs/loader.js:769:14)
    at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:72:12)
    at internal/main/run_main_module.js:17:47

neon ERR! spawn cargo ENOENT

Error: spawn cargo ENOENT
    at Process.ChildProcess._handle.onexit (internal/child_process.js:269:19)
    at onErrorNT (internal/child_process.js:465:16)
    at processTicksAndRejections (internal/process/task_queues.js:80:21)
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] install: `neon-load-or-build`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the [email protected] install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

[black-puppydog]

okay I feel like there's some assumption of mine that's not holding up.

Error: No native build was found for platform=linux arch=arm64 runtime=node abi=83 uv=1 armv=8 libc=glibc node=14.15.0

from my understanding, this line should only show up if you're trying to compile a version that includes the new neon/rust version of ssb-keys.
I think what may be happening is that you're actually still on a commit that includes those. That may be due to the sequence in which you did the different git checkout and git switch commands and such.

What is your git status at the moment?

[sunjam]

What is your git status at the moment?

$ git status
On branch pre-rust
nothing to commit, working tree clean

[black-puppydog]

sorry, I meant git log HEAD~..HEAD

For comparison, this is mine:

commit 3238adddaa965d1ff8410975ec519f5d4f234e5a (HEAD -> rust-free, tag: pre-rust)
Merge: 9d3091b4 75cea600
Author: Christian Bundy <[email protected]>
Date:   Tue Sep 8 09:38:08 2020 -0700

    Merge pull request #1315 from ssbc/conn0181

    update ssb-conn to 0.18.1

commit 75cea600bb697c1373eec470224a32da87d5befe
Author: Andre Staltz <[email protected]>
Date:   Tue Sep 8 10:39:02 2020 +0300

    update ssb-conn to 0.18.1

[black-puppydog]

(the git log command shows the commit hashes, which is a unique way of seeing where we are. branch/tag names can be ambiguous)

[sunjam]

(the git log command shows the commit hashes, which is a unique way of seeing where we are. branch/tag names can be ambiguous)

$ git log HEAD~..HEAD
commit 54a329a6358694d3f586ed419a6c6aad6f5beecf (HEAD -> pre-rust, origin/master, origin/HEAD, master)
Merge: 810bb6a7 88d5d176
Author: Daan Wynen [e-mail]
Date: Sat Nov 14 17:11:10 2020 +0100

Merge pull request #1327 from fabiocosta0305/master

Some corrections on Brazilian Portuguese translation.

commit 88d5d1760daa915bd4202419ea5123d0d9a5b27e
Author: Fábio Emilio Costa [e-mail]
Date: Sat Nov 14 13:06:04 2020 -0300

Some corrections on Brazilian Portuguese translation.

[black-puppydog]

Ooookay, I think that's progress! :tada:

Here's what I think happened:
further up, you did a checkout of the pre-rust branch instead of the tag git checkout -b pre-rust.
From that moment on, you had a local pre-rust branch, and when you did git checkout pre-rust (without the "-b") later it didn't actually switch to the right commit (the one tagged by me) but checked out the now-existing branch, which was created at the wrong commit. So all your previous builds were actually still trying to compile the version of patchwork that includes the neon patches, which won't yet work on arm.

So, what I think should fix this:

# reset all git changes (locales or whatever) just to make sure the following git commands don't complain
git reset --hard

# check out master branch so we know where we are
git checkout master

# delete the erroneous pre-rust branch
git branch -D pre-rust

# now check out the pre-rust TAG (note absence of "-b" here!)
git checkout pre-rust

# git should have complained about being in "detached" mode. 
# the next commit "fixes" that (but isn't strictly necessary)
# note that I'm using a different name for the branch, to avoid confusion with the tag pre-rust
git switch -c rust-free

# switch to latest LTS node version and associated npm version
nvm use --lts

# clean node_modules. probably not necessary but for good measure...
rm -rf node_modules

# build
npm ci

# launch. -- --no-sandbox may or may not be necessary, depending on your distro. if it is, note the doubled "--"
npm start -- --no-sandbox

[black-puppydog]

Did that solve it? :)

In any case, I'll keep working on getting the build environment correct on arm, too. This is all new for me though, so I don't have a timeline on this, I'm just doing that because I'll soon be running more on arm myself. 😛

[sunjam]

Yep, solved. Thanks for working in this. Maybe there will be arm64
releases for Patchwork someday. Glad to be back and scuttling.

On Mon, Nov 16, 2020, 2:06 AM Daan Wynen [e-mail] wrote:

Did that solve it? :)

In any case, I'll keep working on getting the build environment correct on
arm, too. This is all new for me though, so I don't have a timeline on
this, I'm just doing that because I'll soon be running more on arm myself.
😛

—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
https://github.com/ssbc/patchwork/issues/1326#issuecomment-727875703,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AANUKZSUQBDHSPYPWNQOFFLSQD2SLANCNFSM4TU7DCLA
.