Node-sqlite3: Automated ARM builds

[coolaj86]

I would be happy to build arm6 and arm7 bins for both sqlite3 and sqlcipher.

v0.10 and v1.2 work

v0.11 and <= v1.1 fail

v0.12 I haven't tested.

I have internet-accessible PIs connected to a few routers that I can offer for such a task.

[springmeyer]

We build all binaries on travis (for linux and mac) and appveyor (for windows). What would be very useful is a travis job that used qemu or something like it to build rpi binaries. @bander9289 has been experimenting with this (and I've been meaning to learn but not had time yet): https://github.com/mapnik/mapnik-packaging/compare/master...bander9289:master#diff-7142a27433410bb3f7e3ae34a1674807R16

@coolaj86 - interested in providing a pull request adding rpi build support to travis.yml?

/cc @mojodna who may also be interested in this effort.

[mojodna]

Hmm. Interested I am, yet not yet needing it. Thanks.

[coolaj86]

My work on embedded systems has lead me to believe that emulation is no substitution for compiling on the real deal, so I wait the 15+ minutes out when I need it.

I'm open to trying something like Qemu, but it would probably be easier for me to set up a Pi as a build server than to learn Qemu and what special compile options I'd need to pass and such.

What are the advantages of Qemu that make you prefer that solution?

[alexellis]

@coolaj86 does it not come down to automation, automation, automation and repeatability? If I were to take a binary for sqlite3 instead of source, then I'd definitely want it to have been built through the 'trusted' channels - not a third party. Having said that something really needs to be done #597

[marvinroger]

Scaleway baremetals ARMv7 servers, might help here. Each build, assuming it would last less than 59 minutes, would cost €0.006.

Moreover, with their CLI, create, start and SSH into a server is as easy as:

$ scw exec $(scw start --wait $(scw create ubuntu-trusty)) /bin/bash
[root@noname ~]#

Note I am not affiliated to them in any way, that's just how I handle my personal Docker builds. 😉

[kkaefer]

Note that we don't have build on ARMv7; instead we could also cross-compile.

[alexellis]

The build takes a ridiculous amount of time to complete especially on the Pi Zero - that I'd give up on sqllite or use python. Please provide a binary through npm to support continued use.

[skl]

I've automated an ARM build with Vagrant, Docker (as opposed to chroot in @bander9289's approach) and QEMU which is one option for an overall npm install use case for ARM targets - whilst being able to build on a more powerful machine. I haven't open sourced it yet but this is the general idea that I use for sqlite3 and bcrypt:

1. Spin up a single-core vagrant box with "debian/jessie64"
2. Install docker-engine and qemu-user-static in the vm
3. Spin up a docker container with docker run -it --volume /usr/bin/qemu-arm-static:/usr/bin/qemu-arm-static philipz/rpi-raspbian bash
4. Install build-essential and nodejs in the container
5. npm install inside the container

I was not able to get a full QEMU ARM VM running but I did have success with the QEMU User Mode emulation as above. So it runs only the docker process under ARM emulation rather than providing a full virtual machine.

One major drawback of either QEMU approach is that is only supports a single thread, so for example I could not install Git LFS within the ARM docker container as Go binaries cannot be run through QEMU due to the concurrency aspects of the language. So my advice is do as much as you can within the "host VM" then delegate the ARM portions to the container.

Maybe some of the above could help for the Travis build? Cross-compilation would be ideal but it didn't seem straight forward to me, that's why I went the QEMU route instead as it also covers other npm modules at the same time.

[StefanScherer]

@skl What about using Travis or at least using the steps described here: https://blog.hypriot.com/post/setup-simple-ci-pipeline-for-arm-images/

[skl]

Thanks for the article, uses the same principal and applies it to Travis. Just in my case I need private builds for work.

[springmeyer]

Anyone want to try getting qemu running on travis (within docker as a bonus) so we can start building and publishing binaries for ARM/rasp?

[thom-nic]

Cross-post from #996: this comment looks like it has a workable path for an ARM environment on Travis: https://github.com/travis-ci/travis-ci/issues/3376#issuecomment-168003756 similar to @skl's solution, except it looks like a "docker in docker." Have not tested myself.

[rzr]

An other option would be to submit a deb package of node-sqlite3 to debian then use if on raspbian...

Well I can try to hack to Docker file to build package.

Edit:

https://tracker.debian.org/pkg/node-sqlite3

while not in Raspbian yet ?

http://archive.raspberrypi.org/debian/pool/main/n/

[rzr]

@springmeyer would that help ? then we can tune travis for it mentoring welcome

[rzr]

Do you then want to do continuous delivery ? if yes can you recommend a service to host built packages ?

[tkurki]

This works on my OS X machine: compiles and produces a working (tested only under emulation) node-sqlite3 install on ARM

docker run -it arm32v7/node:8-jessie bash
npm install sqlite3

The compile outputs a bunch of warning and takes for ages, but seems to work.

See https://hub.docker.com/r/arm32v7/node for the different versions available.

This does not work out of box on Travis and services (docker) are not supported on OS X there. Does somebody have the fu to get this working on Travis? I have been able to build arm32v7 Docker images there, but this escapes me.

[StefanScherer]

@tkurki Happy to help. We did Travis CI builds for our ARM images and wrote a blog post about it https://blog.hypriot.com/post/setup-simple-ci-pipeline-for-arm-images/
The trick is to enable QEMU (which is enabled by default in Docker Desktop for Mac and Windows, maybe by our demand ;-) ).
Let me know if you need more details.

[tkurki]

Ok, here's basically the same thing, running npm install sqlite3 in an emulated ARM container: https://github.com/tkurki/node-sqlite3/commit/01a783e0f500182449347ed9b908e79d716de5f9 and it works on Travis.

Once the container is built you could start it with a directory mounted from the build host and copy the artifacts to the host.

Would this work?

[StefanScherer]

Yes this should work. When you can build an image each RUN instruction is also a short lived container. A docker run -v should work just the same.

[StefanScherer]

But you can also use docker create to just create a container and docker cp from that container back to the host.

[tkurki]

Thanks for the tip - docker cp `docker create <image>`:<full-path-to-file> . works like a charm!

Now here is a branch that runs npm install sqlite3 in an ARM container with Travis and then copies the resulting .../node-v57-linux-arm/node_sqlite3.node to the build host. The result looks like the real thing afai can tell (did not try it on my Pi):

$ file node_sqlite3.node
node_sqlite3.node: ELF 32-bit LSB  shared object, ARM, EABI5 version 1 (SYSV), dynamically linked, BuildID[sha1]=1f872eb22114592ad62b20cd85746d8ec95a39ca, not stripped

Maybe somebody more versed with how the prebuilt binaries end up where they are supposed to could take this from here?

[rzr]

Let's split the feature in several patches: 1st start with:
https://github.com/mapbox/node-sqlite3/pull/1112

[kewde]

recommend a service to host built packages

There are build scripts that will automatically detect the architecture, version and upload to the S3 bucket.
You do however need to pass the environment variables from Travis to the container.

[rzr]

Today, I had a look at this but I am currently blocked by:

https://github.com/nodejs/node/issues/26353#

[tkurki]

Hmm. Isn't this actually a lot simpler?

You don't have to build a Docker container, if you have a prebuilt Node container for the target architecture.

For example on a Docker host that has bin_fmt support for ARM (like my Macbook) you can build & publish node-sqlite3 with a single command:

docker run -it --rm -v "$(pwd)":/build arm32v7/node:8 bash -c  'cd /build && npm install --unsafe-perm --build-from-source && node_modules/.bin/node-pre-gyp package testpackage publish info'

I don't have the credentials for the s3 upload, so naturally publish fails for me, so this is not tested end to end.

So: if we add something like

#!/usr/bin/env bash

docker run --rm --privileged multiarch/qemu-user-static:register --reset
mkdir tmp && pushd tmp && curl -L -o qemu-arm-static.tar.gz https://github.com/multiarch/qemu-user-static/releases/download/v2.11.0/qemu-arm-static.tar.gz && tar xzf qemu-arm-static.tar.gz && popd
docker run -it --rm -v "$(pwd)":/build arm32v7/node:8 bash -c  'cd /build && npm install --unsafe-perm --build-from-source && node_modules/.bin/node-pre-gyp package testpackage publish info'

as scripts/build_arm_with_docker.sh, add Docker support to .travis.yml and trigger the script from Travis build I believe we have a pretty elegant way to produce prebuilt ARM binaries!

_Look, ma', no Dockerfile needed._

[tkurki]

See #1149 for a working example of the idea in the previous message.

[xPaw]

1147 was merged, is there any blocker on hooking that up to the CI?

[n1ru4l]

I created https://github.com/mapbox/node-sqlite3/pull/1362 which seems a bit less complicated and also provides alpine binaries. The only thing missing is anyone reviewing it and a repo maintainer setting S3 bucket credentials.