Prisma: Provide binaries for Alpine Linux

[steebchen]

Currently, running Prisma on Alpine Linux fails, because the libssl version does not match.

For more information about this error, see https://github.com/prisma/photonjs/issues/173

Steps needed to make this work:

• [x] Build alpine-specific binaries.
  We currently provide alpine binaries but it seems they're very unstable, so we have to revisit this and best built the alpine build process from scratch. This involves a lot of debugging and finding out if we need multiple libssl variations.
• [x] Find out how to install libssl on the user's machine easily so it can be documented.
• [ ] Adapt the fetch-engine in photonjs once the binaries are published.

[janpio]

Internal Note: Find out what that actually means and ask @mavilein if this should be implemented.

[LongLiveCHIEF]

related #938

[LongLiveCHIEF]

I'm currently working on this as part of #938. The best route to take would be to provide a repository for each distro you support, and in your package manifest for that distro, you would declare what dependencies your package has, so that the package manager can find the best match.

Then the user can use the native package manager to to determine the best way to install packages in a way that is compatible with their platforms internet and security policies.

My suggestion is to support package channels/repositories for the following formats:

| Format | Family [Distros] | Package Manager |
| --------- | --------------------- | ------------------------- |
| .apk | Alpine Linux | apk |
| .deb | Debian [Ubuntu, Raspbian, Kali, PureOS, Tails] | apt | dpkg |
| .rpm | RHEL [ RHEL, CentOS, Fermi, Fedora, Atomic] | yum |

[janpio]

Hey @LongLiveCHIEF, although I understand where you are coming from this seems out of scope of what @steebchen was describing here - and what will be implemented as first step to get Prisma running on Alpine Linux - but justifies you opening a new issue, as it raises valid points and might be a good idea. Can you please do that? Thanks.

[matthewmueller]

We don't have the resources to tackle this right now, but if you're blocked on this please let us know and we'll re-prioritize!

[dpetrick]

Rust with Musl libc is not straight forward.

[tvvignesh]

Just did a container scanning in Google Cloud. Looks like using the Node base image is bringing in a lot of vulnerabilities :eyes: Please let me know when you are there and I can be your alpha user :sweat_smile:

Tried changing it to alpine and I got this:

Also, saw this article: https://snyk.io/blog/take-actions-to-improve-security-in-your-docker-images/

[tvvignesh]

Btw, looks like node-rdkafka had similar issues (https://github.com/Blizzard/node-rdkafka/issues/361) and they have a solution here: https://github.com/Blizzard/node-rdkafka/blob/master/examples/docker-alpine.md

I tried it but somehow, it didn't work. Maybe, I am doing it wrong.

[tvvignesh]

Nvm, As @olup suggested here: https://github.com/prisma/photonjs/issues/173#issuecomment-518013542 this works for me.

Used this as base image:

FROM frolvlad/alpine-glibc
RUN apk update && apk add nodejs npm openssl python g++ make && rm -rf /var/cache/apk/*

But since it installed Node v10.x and was using older alpine image, did some googling and this is now my base image:

FROM alpine:3.11.2

ENV LANG=C.UTF-8
ENV GLIBC_VERSION 2.30-r0

# Download and install glibc
RUN apk add --update curl && \
    curl -Lo /etc/apk/keys/sgerrand.rsa.pub https://alpine-pkgs.sgerrand.com/sgerrand.rsa.pub && \
    curl -Lo glibc.apk "https://github.com/sgerrand/alpine-pkg-glibc/releases/download/${GLIBC_VERSION}/glibc-${GLIBC_VERSION}.apk" && \
    curl -Lo glibc-bin.apk "https://github.com/sgerrand/alpine-pkg-glibc/releases/download/${GLIBC_VERSION}/glibc-bin-${GLIBC_VERSION}.apk" && \
    apk add glibc-bin.apk glibc.apk && \
    /usr/glibc-compat/sbin/ldconfig /lib /usr/glibc-compat/lib && \
    echo 'hosts: files mdns4_minimal [NOTFOUND=return] dns mdns4' >> /etc/nsswitch.conf && \
    apk del curl && \
    rm -rf glibc.apk glibc-bin.apk /var/cache/apk/*


RUN apk add nodejs npm python g++ make && rm -rf /var/cache/apk/*

Taken from https://github.com/jeanblanchard/docker-alpine-glibc

[radicand]

For anyone wanting to use Alpine currently, you can use Cross: https://github.com/rust-embedded/cross to compile binaries. I do this for arm64 and x86_64 now by using the $arch-unknown-linux-musl target. The latest images from this project have removed openssl, and will not build prisma properly, but if you use the 0.1.6 images, it works as expected. Due to #1722 there is a lot of workaround required to copy the binaries to expected locations at runtime, but once this is resolved it should get easier. That said, I'd love to see Alpine binaries provided natively, and for arm64 as well (personal wish).

To cross-compile yourself easily:

1. Ensure you have Rust setup for your environment. Docker is also a prerequisite.
2. Clone the prisma-engines repo. You'll want to match the git commit you use with your version of prisma2 in use. HEAD is always moving and may have breaking changes from one of the published previews. I've matched this by installing the preview module on my local machine and doing query-engine --help which prints the commit hash used at the top. Commands and YMMV.

3. Create a Cross.toml file containing:

   [target.aarch64-unknown-linux-musl]
   image = "rustembedded/cross:aarch64-unknown-linux-musl-0.1.16"
   passthrough = [
       "SERVER_ROOT", "RUST_LOG_FORMAT", "RUST_BACKTRACE", "RUST_LOG", "LOG_QUERIES", "LOG_LEVEL", "TEST_MODE",
       "PRISMA_CONFIG_PATH", "PRISMA_DML_PATH", "PRISMA2_BINARY_PATH", "PRISMA_BINARY_PATH",
       "MIGRATION_ENGINE_BINARY_PATH", "SQLITE_MAX_VARIABLE_NUMBER","PKG_CONFIG_ALLOW_CROSS"
   ]
   
   [target.x86_64-unknown-linux-musl]
   image = "rustembedded/cross:x86_64-unknown-linux-musl-0.1.16"
   passthrough = [
       "SERVER_ROOT", "RUST_LOG_FORMAT", "RUST_BACKTRACE", "RUST_LOG", "LOG_QUERIES", "LOG_LEVEL", "TEST_MODE",
       "PRISMA_CONFIG_PATH", "PRISMA_DML_PATH", "PRISMA2_BINARY_PATH", "PRISMA_BINARY_PATH",
       "MIGRATION_ENGINE_BINARY_PATH", "SQLITE_MAX_VARIABLE_NUMBER","PKG_CONFIG_ALLOW_CROSS"
   ]
   

4. Source the env file as stated in the prisma-engine build instructions: source .envrc

5. Install Cross: cargo install cross
6. Compile the binaries (this will take some time): PKG_CONFIG_ALLOW_CROSS=1 cross build --release --target=x86_64-unknown-linux-musl where target is your architecture. ARM64 would be PKG_CONFIG_ALLOW_CROSS=1 cross build --release --target=aarch64-unknown-linux-musl by way of example.
7. Copy the compiled binaries from ./target/x86_64-unknown-linux-musl/release/ (update with the architecture you used). You'll need to rename and copy the binaries manually per my note above regarding ENV variables being broken right now.

HTH

[sunil-sadasivan]

Looks like @tvvignesh solution stopped working with the latest prisma/cli release. This was a viable solution for me to run prisma on Alpine however now I get this error when running prisma generate

Error: Precompiled binaries are not available for Alpine.

This runs fine on @prisma/[email protected] so for now I've forced installed this verison.

[tvvignesh]

@sunil-sadasivan It works even with the latest release. I use it daily (currently i am on 2.0.0-beta.2)

I run generate like this within the docker file: npx prisma generate --schema \"/path/to/schema/schema.prisma\"

[sunil-sadasivan]

@tvvignesh hmm, the latest I believe is 2.0.0-beta.3 which is where I'm able to reproduce the issue.

[tvvignesh]

@sunil-sadasivan Ahh. Haven't checked with beta.3 I will check tomorrow and get back to you.

Related: https://github.com/prisma/prisma/issues/1635

[radicand]

Temporary alternative: I've been building binaries for Alpine for awhile now, you can grab them from my Gitlab pipeline here: https://gitlab.com/npappas/prisma-engines/pipelines, find the version you need and download the artifacts from the most recent build on that version. There are binaries for amd64 and arm64 included. You'll need to copy them in as part of your image build, and then use the ENV variables to tell prisma where the binaries are. I have beta.3 building now, it'll be done in about an hour.

[radicand]

Having had a chance to use the newly built beta3 binaries, I can confirm @sunil-sadasivan 's issue - there was a change made in #1635 where the error is dumped out for Alpine, regardless of it valid binaries exist, or the platform can run stock ones. I opened a new issue to track this: #2266

[tvvignesh]

While I am about to check this issue from my end, looks like the changes @Jolg42 described here https://github.com/prisma/prisma/issues/1635#issuecomment-613938112 breaks the Alpine builds as @sunil-sadasivan and @radicand have mentioned. Any alternative? It used to work till beta.2 and looks like explicit checking has been added which breaks this. Will check from my end and report back anyways.

I see this is the line where you are throwing the error: https://github.com/prisma/prisma/blob/c86af74877d9f39936b18276b3c1db8d5ff7b310/src/packages/fetch-engine/src/download.ts#L74-L80

I also see a change in binary from prisma to query-engine here: https://github.com/prisma/prisma/issues/1920 - have to check if any of it has any implications for alpine users.

Not sure what other changes would impact this. Any ideas?

CC: @janpio @timsuchanek

[Jolg42]

Thanks @tvvignesh so the error thrown was intended to make it easier for people to know that Alpine was not supported. (The previous error message was not helpful about that).

It was not intended to block you from using Alpine, so this will be changed to a warning instead 😅

This is tracked here https://github.com/prisma/prisma/issues/2266 and was just closed now 🎊

[tvvignesh]

Great! Will test it out today and get back. Btw, is it released in alpha? Or is it better to wait for the next beta?

[Jolg42]

@tvvignesh It is indeed in the latest alpha since 2.0.0-alpha.1121

So you can already use it locally, I won't recommend to use it in production because the alpha channel is not as stable. The next beta is coming soon 😉

[radicand]

I updated my Dockerfile this morning to streamline how I integrate the binaries I build into my Docker images - see https://gitlab.com/recipe-cookbook/backend/-/blob/3e4b7005d261a683919ebd09e03bbbfdad88a6a1/Dockerfile for an example. Available architectures are x86_64 and arm64.

If you choose to use this approach, just keep your prisma version in sync with the build artifact links available at https://gitlab.com/npappas/prisma-engines/pipelines

Still hoping we'll get natively built binaries some day.

[timsuchanek]

Thanks a lot for reporting 🙏
This issue is fixed in the latest alpha version of @prisma/cli.
You can try it out with npm i -g @prisma/cli@alpha.

In case it’s not fixed for you - please let us know and we’ll reopen this issue!

[Jolg42]

No need to compile custom binaries for Alpine anymore :confetti_ball:

[pantharshit00]

No need to install glibc in your alpine images now, we ship a binary compatible with musl libc now

[radicand]

I can confirm this works on alpha - thanks everyone for making this happen! For anyone tracking, I'll still be building binaries for alpine/arm if/until that happens down the road.