Zwave2mqtt: [feat] NodeJS 12.x (Erbium)

[frenck]

Is your feature request related to a problem? Please describe.

I've seen to be unable to get NodeJS 12.x (Erbium) working with Zwave2Mqtt.
I also noticed the Docker images are still using NodeJS 8.x (Carbon), which is EOL and unsupported as of 2019-12-31.

Describe the solution you'd like

Upgrade to NodeJS 12.x (Erbium)

Describe alternatives you've considered

n/a

Additional context

Initial blocker: node-serialport, needs to be at least 7.1.5 (7.1.3 right now).

[robertsLando]

@frenck Thanks for the issue, fixed now!

[robertsLando]

@frenck About nodejs 8 in docker containers, I think I will keep it for long time (until there would be a real advantage for it) because the build process uses pkg to build application binaries in order to keep the docker image as clean as possible, unfortunally pkg doesn't provide nodejs binaries for arm64 and that is why I'm using my own compiled binaries for arm64 builds of docker. I would need to create binaries for nodejs 12 for arm64 and than test that everything works (would me take at least one day) and there is no reason to do it right now

[frenck]

Sorry, that statement makes no sense to me.

There is no reason to use pkg, nor is there a dependency on this project to work.
That is a personal choice, where you choose to stick the project on an old, non-supported version.

For contrast, the Hass.io add-on does not use it, supports 5 different architectures (including arm64/aarch64) and is on Node 12.x, based on Alpine 3.11. Remaining the same size as your pkgs based Docker images, while including additional packages for Hass.io as well).

[robertsLando]

Sincerly I don't remember the reason why I decided to use pkg now but I could try to fix this too in next days, also builds will take less time to complete. Thanks :)

[robertsLando]

Hi @frenck. As you suggested I have updated docker repo to use nodejs erbium in build process. The problem is that I'm unable to compile arm images as I keep getting this error when istalling serialport.

I have tried everything but I'm stucked. Do you have any suggestion? This is the Dockerfile I'm using:

# ----------------
# STEP 1:
# Build Openzwave and Zwave2Mqtt pkg
# All result files will be put in /dist folder
FROM arm32v6/node:carbon-alpine AS build

COPY bin/qemu-arm-static /usr/bin/

# Set the commit of Zwave2Mqtt to checkout when cloning the repo
ENV Z2M_VERSION=ba709b0a6b52b3d2c3a84072d90b2b654626de8e

# Install required dependencies
RUN apk update && apk --no-cache add \
      gnutls \
      gnutls-dev \
      libusb \
      eudev \
      # Install build dependencies
    && apk --no-cache --virtual .build-deps add \
      coreutils \
      eudev-dev \
      build-base \
      git \
      python \
      bash \
      libusb-dev \
      linux-headers \
      wget \
      tar  \
      openssl \
      make 

# Get latest stable 1.6 and move binaries in /dist/lib and devices db on /dist/db
RUN cd /root \
    && wget http://old.openzwave.com/downloads/openzwave-1.6.1004.tar.gz \
    && tar zxvf openzwave-*.gz \
    && cd openzwave-* && make && make install \
    && mkdir -p /dist/lib \
    && mv libopenzwave.so* /dist/lib/ \
    && mkdir -p /dist/db \
    && mv config/* /dist/db

# Clone Zwave2Mqtt build pkg files and move them to /dist/pkg
RUN cd /usr \
    && git clone https://github.com/OpenZWave/Zwave2Mqtt.git  \
    && cd Zwave2Mqtt \
    && git checkout ${Z2M_VERSION} \
    && npm config set unsafe-perm \
    && npm install \
    && npm run build \
    && mkdir -p /dist/app \
    && mv /usr/Zwave2Mqtt/* /dist/app

# Clean up
RUN rm -R /root/* && apk del .build-deps

# ----------------
# STEP 2:
# Run a minimal alpine image
FROM arm32v6/node:carbon-alpine

COPY bin/qemu-arm-static /usr/bin/

LABEL maintainer="robertsLando"

RUN apk update && apk add --no-cache \
    libstdc++  \
    libgcc \
    libusb \
    tzdata \
    eudev 

# Copy files from previous build stage
COPY --from=build /dist/lib/ /lib/
COPY --from=build /dist/db/ /usr/local/etc/openzwave/ 
COPY --from=build /dist/app/ /usr/src/app/

# Set enviroment
ENV LD_LIBRARY_PATH /lib

WORKDIR /usr/src/app

EXPOSE 8091

CMD ["node", "bin/www"]

[frenck]

First glance, you need python2-dev to be able to build it. (you have python which itself is not needed, and I believe it pulls in Python 3 right now).

Here is the add-on build file, This single file is used for armhf, armhf, aarch64, amd64 and i386 builds:

https://github.com/hassio-addons/addon-zwave2mqtt/blob/master/zwave2mqtt/Dockerfile

I've just updated the add-on to v2.1.0, and all builds completed successfully:

https://gitlab.com/hassio-addons/addon-zwave2mqtt/pipelines/108321437

[robertsLando]

@frenck My script was working until last edit but I just changed the base image from carbon to erbium.

Even If I set pyton to 2.7 and I go back to carbon the error is the same. Any other thoughts?

That error is really strange...

[frenck]

I've copied your Dockerfile (besides the $$ -> && on the npm config set unsafe-perm line).

It just builds fine. Did an armv6 & armv7 build (on native systems, I don't use Qemu).

[robertsLando]

I think it is something related to qemu so :( Just curious, how do you handle the cross-arch builds? The only way I found is using qemu and than I have a build scripts that creates the manifest but that has been a pain to create... i wish there would be a service almost automatic for doing this. Can you point me to a documentation or what else you are using for builds?

P.S: Thanks for the test you did for me :handshake:

[frenck]

Could be indeed. Sometimes trying a different Qemu version helps. e.g., I had troubles with QEMU 4.x (including performance drops). https://github.com/hassio-addons/qemu-user-static/releases So 3.1.1 for me right now.

(Register in Docker before build: docker run --rm --privileged hassioaddons/qemu-user-static:latest)

That said, I include Qemu in my base images (for others to use), but don't build emulated, but native only. I have a server farm with different architectures (including some beefy Pine64 Rock64Pro's and things like an Odroid N2, those can handle 32 bits arm natively on the CPU).

I use GitLab-CI at this moment (Not sure if I would do that again if I had to choose now, I might need to re-evaluate that). With that I'm able to send specific architecture builds out to their matching runnings on native capable machines.

This removed a lot of frustration, in terms of emulation issues and build speed performance (e.g, the above Dockerfile of yours took just a couple of minutes to build on a Pine64).

[frenck]

In case you are interested, this is the actual build template I use for all Hass.io Community add-on builds (with 5 different archs): https://github.com/hassio-addons/organization/blob/master/gitlabci/addon.yml#L79

[robertsLando]

Yep I saw that on GitLab but I have no idea from where to start. I wish docker hub creates something for this in the future... Docker is still a pain for cross-arch builds

(e.g, the above Dockerfile of yours took just a couple of minutes to build on a Pine64).

It takes me at least 30 minutes to build with qemu :cry:

Thanks for sharing this info @frenck :)

[robertsLando]

In case you are interested, this is the actual build template I use for all Hass.io Community add-on builds (with 5 different archs): https://github.com/hassio-addons/organization/blob/master/gitlabci/addon.yml#L79

Thanks for this but I think I would never take this approach It seems more complicated then the one I'm using now :laughing:

[billimek]

Is TravisCI an option, they allegedly support arm-based builds now: https://blog.travis-ci.com/2019-10-07-multi-cpu-architecture-support

[robertsLando]

@billimek Travis is my first alternative, I think it could became one of the main services for cross arch building but didn't know since now about the possibility to test it cross archs :heart_eyes:

Thanks @billimek for pointing this out, will give it a shot for sure once I found the time

[robertsLando]

@billimek Actually seems that only amd64 and arm64 archs are supported :(

[Fishwaldo]

For local builds you can try Docker buildx (I think some of the CI platforms support it as well like github actions etc).

You have to enable experimental features in docker to use it but it worked (although slow) for me.

[Fishwaldo]

https://docs.docker.com/buildx/working-with-buildx/

[robertsLando]

@Fishwaldo I know that but it's almost like the method I'm using right now, it does the same, I'm looking for an online service that makes native cross-arch builds.

Also manifest management is still a pain (you still need to enable exeprimental cli to use it). This is frustating for everyone that uses Docker to develop cross-arch applications

[Fishwaldo]

I searched for native armv7 build options. Only ones I found were paid for. 

The buildx uses a custom build of QEMU, and emulates 1 level higher than what your trying. Maybe worth a shot? 

On 11 Jan 2020, at 12:01 AM, Daniel Lando [e-mail] wrote:



@Fishwaldo I know that but it's almost like the method I'm using right now, it does the same, I'm looking for an online service that makes native cross-arch builds.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.