Yarn: Publish Yarn to the Alpine Linux package repository (APK)

[ide]

This is a feature request to publish Yarn to Alpine Linux's package repository. The primary motivation is to be able to more easily use Yarn when building Docker images built on top of Alpine Linux. (Alpine is a distro that's much lighter than Ubuntu and earlier this year there was some discussion that Docker was moving its official images to Alpine: https://www.brianchristner.io/docker-is-moving-to-alpine-linux/.)

Feb 2017 Edit: Following the discussion below, if you are running Alpine in a Docker image that already includes Node (ex: mhart/alpine-node:base-7), you might want to install Yarn by copying the files directly:

ENV YARN_VERSION 0.20.0
ADD https://yarnpkg.com/downloads/$YARN_VERSION/yarn-v${YARN_VERSION}.tar.gz /opt/yarn.tar.gz
RUN yarnDirectory=/opt/yarn && \
    mkdir -p "$yarnDirectory" && \
    tar -xzf /opt/yarn.tar.gz -C "$yarnDirectory" && \
    ln -s "$yarnDirectory/dist/bin/yarn" /usr/local/bin/ && \
    rm /opt/yarn.tar.gz

[Daniel15]

Hi James! 😄

Looks like fpm added support for Alpine Linux packages. We're already using fpm to build the CentOS/RHEL .rpm, so I should be able to build an Alpine Linux package too.

[SimenB]

Any news here? There will be official Alpine images for node shortly (docker-library/official-images#2334), being able to install a self-contained yarn into that would be sweet

[Daniel15]

I'll try to take a look this weekend.

It looks like Alpine only has Node.js 6.7.0 at the moment: https://pkgs.alpinelinux.org/package/v3.4/main/x86_64/nodejs. I guess that's fine, I wonder how well-maintained the Node.js packages are though.

[ide]

https://github.com/mhart/alpine-node is a popular image that is regularly maintained. These Alpine images already have Node installed, apk add yarn just needs to install Yarn -- if it pulls in an extra copy of Node that defeats the purpose of using a small image like Alpine.

[SimenB]

I'd prefer it if it just installed yarn yes, as it will only be used where Node is already available

[Daniel15]

Sure, I can do that. I wonder if Alpine has something like a "recommends" relation between packages. Debian does that - We "recommend" the Node.js package, but people can ignore the recommendation if they have Node.js installed via alternate means (for example, nvm, or compiled from source).

[Daniel15]

Currently blocked by https://github.com/jordansissel/fpm/issues/1227

As an aside:

Alpine is a distro that's much lighter than Ubuntu

Have you considered using Debian? It's also much lighter than Ubuntu.

[CpuID]

Have you considered using Debian? It's also much lighter than Ubuntu.

$ docker images | grep ubuntu
ubuntu    16.04               42118e3df429        4 months ago        124.8 MB
$ docker images | grep debian
debian     jessie              1b088884749b        6 months ago       125.1 MB
$ docker images | grep alpine
alpine      3.4                 4e38e38c8ce0        5 months ago      4.795 MB

Self explanatory.

[lukasoppermann]

Any news here?

[Daniel15]

Currently blocked by the fpm bug linked earlier in the thread.

Sent from my phone.

On Dec 17, 2016 7:49 PM, "Lukas Oppermann" [e-mail] wrote:

Any news here?

—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub
https://github.com/yarnpkg/yarn/issues/1326#issuecomment-267751243, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAFnHTSsEJD0VXXwPKZUsPDjh9dNl7o7ks5rI6IXgaJpZM4Kc3Nl
.

[errm]

I have an open PR to do this https://github.com/alpinelinux/aports/pull/714

[errm]

yarn package is now available in the alpine edge/testing repo
https://pkgs.alpinelinux.org/package/edge/testing/x86/yarn

[silverwind]

Indeed!

echo -e 'http://dl-cdn.alpinelinux.org/alpine/edge/main\nhttp://dl-cdn.alpinelinux.org/alpine/edge/community\nhttp://dl-cdn.alpinelinux.org/alpine/edge/testing' > /etc/apk/repositories
apk add --no-cache yarn

[Daniel15]

@errm Thanks! is it possible to update the package to also patch the installationMethod in package.json? This key tells Yarn how it was installed, so that we can display the correct command to run if someone runs an outdated version of Yarn and we show an upgrade prompt.

[errm]

yes I certainly can add that ...

[Daniel15]

Cool, thanks. I'm going to close this out since the package exists now. Thank you so much, @errm!

[amir20]

I get a strange behavior when doing

FROM mhart/alpine-node:7

RUN echo -e 'http://dl-cdn.alpinelinux.org/alpine/edge/main\nhttp://dl-cdn.alpinelinux.org/alpine/edge/community\nhttp://dl-cdn.alpinelinux.org/alpine/edge/testing' > /etc/apk/repositories \
    && apk add --no-cache yarn

node -v is now 6.9.5. Does anybody else have this problem?

[Daniel15]

@amir20 You didn't mention what the strange behaviour actually is 😛

[amir20]

The strange behavior is that I was expecting node 7 but it's actually 6.9.5. 👎

[ierceg]

@amir20 apk add yarn adds node as dependency:

(18/23) Installing libcrypto1.0 (1.0.2k-r0)
(19/23) Installing http-parser (2.7.1-r0)
(20/23) Installing libssl1.0 (1.0.2k-r0)
(21/23) Installing libuv (1.10.2-r0)
(22/23) Installing nodejs (6.9.5-r0)
(23/23) Installing yarn (0.19.1-r0)

[Daniel15]

Ahh, @amir20, I see what you mean. It looks like mhart/alpine-node installs Node.js from a tarball: https://github.com/mhart/alpine-node/blob/master/Dockerfile#L21. This means that any Alpine packages that depend on Node.js will also end up installing their own separate version of Node.js. You might have better luck using a Docker image that installs Node.js using an Alpine package instead. Using the system's package manager is generally the best way of installing apps on Linux.

[SimenB]

@Daniel15 This is the same way the official Node docker images on Alpine does it. https://github.com/nodejs/docker-node/blob/master/Dockerfile-alpine.template#L31-L43

So a version in the registry that doesn't depend on Node directly would be awesome. I think only that version is needed, but providing both might be nice.

There are no prebuilt binaries of Node for Alpine, the version on there now isn't official. There's a reason it's compiled in the Docker image itself 😄

https://github.com/nodejs/build/issues/75

See earlier discussion in this thread: https://github.com/yarnpkg/yarn/issues/1326#issuecomment-259766931

[m4rcelofs]

Adding node as a dependency causes apk add yarn to almost double the size of images extending the official node:X-alpine ones.

[silverwind]

I wonder if apk could somehow be made aware of a self-compiled node like in @mhart's image.

[Daniel15]

Package management systems work best if every app on the system is managed
by them. Honestly you're going to have far fewer issues if you just use an
Alpine package for Node.js too.

Having said that, for the Debian package, we had to change the node
dependency from a "Depends" relationship to a "Suggests" or "Recommends" (I
forget which) to handle when people use nvm. I wonder if Alpine has
something like that (optional dependencies).

Sent from my phone.

On Feb 2, 2017 8:33 AM, "silverwind" [e-mail] wrote:

I wonder if apk could somehow be made aware of a self-compiled node like
like in @mhart https://github.com/mhart's image.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/yarnpkg/yarn/issues/1326#issuecomment-277008289, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAFnHRabd9M9XsK2dAcfFtw0o_Ut1ECpks5rYgVagaJpZM4Kc3Nl
.

[silverwind]

I think a ideal solution would be that instead of doing make install, one would create a apk package that "provides" and "conflicts" node (I'm using Arch Linux jargon).

[Daniel15]

one would create a apm package that "provides" and "conflicts" node

For which package? If we did that on the Yarn package, it wouldn't work since people wouldn't be able to install Node.js?

[silverwind]

For which package

For the self-compiled node in that container. If a package is marked that way in Arch, for example, and the user wanted to installed node from another source, pacman would ask the user if they want to replace the existing package (because they both provide the same and are in conflict). I'm not sure if apk can do that, though.

[Daniel15]

This sounds reasonable to me.

[amir20]

I remember using apt-get there was a way to skip dependencies. I haven't found a way to do that with apk. What @silverwind suggest sounds fine too.

[Daniel15]

With apt-get you can use --no-install-recommends to avoid recommended packages. So for example, if you want to install Yarn but have Node.js installed some other way (like nvm) you can do apt-get install yarn --no-install-recommends

[mhart]

I tend to agree that it's more straightforward if you want to use apk to install yarn, given it's a collection of Node.js scripts, to let apk do its thing.

If you want to use a custom build of Node.js (as in the Docker node images) – then install yarn manually too – then there's no conflict – either curl -o- -L https://yarnpkg.com/install.sh | sh or unpack https://yarnpkg.com/latest.tar.gz wherever you like

[amir20]

I use the following Dockerfile with curl

FROM mhart/alpine-node:7

RUN apk add --no-cache curl && curl -o- -L https://yarnpkg.com/install.sh | sh
ENV PATH /root/.yarn/bin:$PATH

Hope it helps somebody.

[mhart]

Nice – you could remove curl again in the same RUN command chain if you didn't want it in your image too

[amir20]

Oh yea! Good catch.

[mhart]

Also – if you add gnupg – as in apk add --no-cache curl gnupg && ... – then it should verify the files correctly too.

So basically, like this:

RUN apk add --no-cache --virtual .yarn-deps curl gnupg && \
  curl -o- -L https://yarnpkg.com/install.sh | sh && \
  apk del .yarn-deps

[amir20]

Confirmed this works! Thanks @mhart. It reduced my image size by 5 mb. 👍

[gillesdemey]

FYI if you're not using NPM at all you can use mhart/alpine-node:base-7 for an every smaller image!

And if anyone is using the mhart/alpine-node:6 tags don't forget to add tar to the apk install or it will fail with tar: unrecognized option: strip.

[silverwind]

There's one downside to installing yarn manually like that: It won't be available to anyone except root and only during the build process. Probably won't matter if all you do with it is package installation.

I guess users who rely on the latest stable node version are better off using the base alpine image with the apk packages.

[Daniel15]

There's one downside to installing yarn manually like that: It won't be available to anyone except root and only during the build process.

You could manually extract Yarn wherever you like :) Here's an example Dockerfile:

FROM mhart/alpine-node:base-7
MAINTAINER Daniel Lo Nigro <[email protected]>

ENV YARN_VERSION 0.19.1

ADD https://yarnpkg.com/downloads/$YARN_VERSION/yarn-v${YARN_VERSION}.tar.gz /opt/yarn.tar.gz
RUN cd /opt/ && tar xf yarn.tar.gz && mv dist yarn && rm yarn.tar.gz
ENV PATH $PATH:/opt/yarn/bin/

However, I'd suggest using the Yarn Alpine package instead. If you need a newer version of Node.js, you should follow up with whichever team packages Node.js for Alpine 😃

[silverwind]

@Daniel15: Looking good. My only suggestion would be to symlink the binary to /usr/bin instead of modifying PATH.

[Daniel15]

Good idea :) I'd suggest /usr/local/bin as /usr/bin is usually reserved for stuff managed by the system (that is, installed using the native package manager).

[silverwind]

Okay, this should give a pretty small container with yarn available everywhere:

FROM mhart/alpine-node:base

RUN apk add --no-cache curl && \
  mkdir -p /opt && \
  curl -sL https://yarnpkg.com/latest.tar.gz | tar xz -C /opt && \
  mv /opt/dist /opt/yarn && \
  ln -s /opt/yarn/bin/yarn /usr/local/bin && \
  apk del --purge curl

[Daniel15]

@silverwind I'd suggest locking down the version of Yarn being used (like I did in my example above). Also if you're only using curl to download Yarn, you can instead just use ADD to download it, like in my example.

[silverwind]

I like using the latest one in this case so I don't have to maintain the version. And I'm not using ADD to avoid additional image layers 😉.

[SimenB]

@silverwind If you use the latest version of Docker (1.13), you can do docker build -t node-yarn --squash . Which squashes all layers into the FROM. Which means you don't have to do the massive chains anymore 😄

https://github.com/docker/docker/releases/tag/v1.13.0

[errm]

If you want node7 you can just install the nodejs-current package with apk

[mhart]

@errm will that help the situation with yarn depending on nodejs though? Will apk recognize that nodejs-current is a substitute for nodejs?

[mhart]

@errm oh – indeed it does:

/ # echo -e 'http://dl-cdn.alpinelinux.org/alpine/edge/main\nhttp://dl-cdn.alpinelinux.org/alpine/edge/community\nhttp://dl-cdn.alpinelinux.org/alpine/edge/testing' > /etc/apk/repositories
/ # apk add --no-cache nodejs-current yarn
fetch http://dl-cdn.alpinelinux.org/alpine/edge/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/edge/community/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/edge/testing/x86_64/APKINDEX.tar.gz
(1/9) Installing ca-certificates (20161130-r0)
(2/9) Installing libcrypto1.0 (1.0.2k-r0)
(3/9) Installing libgcc (6.3.0-r1)
(4/9) Installing http-parser (2.7.1-r0)
(5/9) Installing libssl1.0 (1.0.2k-r0)
(6/9) Installing libstdc++ (6.3.0-r1)
(7/9) Installing libuv (1.10.2-r0)
(8/9) Installing nodejs-current (7.5.0-r0)
(9/9) Installing yarn (0.20.0-r0)
Executing busybox-1.25.1-r0.trigger
Executing ca-certificates-20161130-r0.trigger
OK: 67 MiB in 20 packages
/ # node --version
v7.5.0
/ # yarn --version
0.20.0

Nice! 👍

[mhart]

I see, because APKBUILD supports a provides declaration:

http://git.alpinelinux.org/cgit/aports/tree/community/nodejs-current/APKBUILD#n17

[chapati23]

Still struggling with this one.

1) Is there any way you can install yarn just once and then cache it on subsequent builds?
2) Right now the "official" way of installing yarn is slower than my hand-baked one:

# Official way
RUN echo -e 'http://dl-cdn.alpinelinux.org/alpine/edge/main\nhttp://dl-cdn.alpinelinux.org/alpine/edge/community\nhttp://dl-cdn.alpinelinux.org/alpine/edge/testing' > /etc/apk/repositories && \
apk add --no-cache yarn

# Hand-baked
ENV YARN_VERSION 0.22.0
ENV YARN_DIR /opt/yarn
ADD https://yarnpkg.com/downloads/$YARN_VERSION/yarn-v${YARN_VERSION}.tar.gz /opt/yarn.tar.gz
RUN mkdir -p $YARN_DIR && \
    tar -xzf /opt/yarn.tar.gz -C $YARN_DIR && \
    rm /opt/yarn.tar.gz

[Daniel15]

Is there any way you can install yarn just once and then cache it on subsequent builds?

Depends on how you're running your builds. I'd suggest having a Docker image with everything required for your build, and then use that as part of your build system. That way, all dependencies are already preinstalled.

Right now the "official" way of installing yarn is slower than my hand-baked one

If you're using Docker, you should only need to install your dependencies once, and then all of your builds should use the resulting Docker image. That means that the slowness of installing Yarn should only be an issue once (or rather, once for every time you update the Dockerfile). You'll likely need to publish the Docker image to the Docker Hub to use it.

Also your hand baked once isn't performing any integrity checking via GPG - You should at least add that. The GPG key is available at https://dl.yarnpkg.com/debian/pubkey.gpg and the signature is at https://yarnpkg.com/downloads/$YARN_VERSION/yarn-v${YARN_VERSION}.tar.gz.asc. You can see how the installation script does it for ideas: https://github.com/yarnpkg/website/blob/master/install.sh#L46

[chapati23]

@Daniel15 thanks, very helpful and good point on integrity checking. i've reverted back to the official way for now.

last 2 things i'm not sure about is handling the yarn.lock file and the yarn cache. is this the way to go: https://hackernoon.com/using-yarn-with-docker-c116ad289d56 ?

seems pretty complex compared to just using good old npm

[Daniel15]

seems pretty complex compared to just using good old npm

I don't think handling the Yarn cache would be any more difficult than handling the npm cache... I'm not familiar enough with Docker to know the best way to handle Yarn's cache.

[whitecolor]

I'm running on mhart/alpine-node:7.1.0

/ # echo -e 'http://dl-cdn.alpinelinux.org/alpine/edge/main\nhttp://dl-cdn.alpinelinux.org/alpine/edge/community\nhttp:/
/dl-cdn.alpinelinux.org/alpine/edge/testing' > /etc/apk/repositories
/ # apk add --no-cache yarn
fetch http://dl-cdn.alpinelinux.org/alpine/edge/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/edge/community/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/edge/testing/x86_64/APKINDEX.tar.gz
(1/6) Installing libressl2.5-libcrypto (2.5.3-r0)
(2/6) Installing ca-certificates (20161130-r2)
(3/6) Installing http-parser (2.7.1-r1)
(4/6) Installing libuv (1.11.0-r1)
(5/6) Installing nodejs (6.10.1-r0)
(6/6) Installing yarn (0.23.2-r0)
Executing busybox-1.24.2-r13.trigger
Executing ca-certificates-20161130-r2.trigger
OK: 49 MiB in 19 packages
/ # yarn
yarn install v0.23.2
info No lockfile found.
[1/4] Resolving packages...
[2/4] Fetching packages...
[3/4] Linking dependencies...
[4/4] Building fresh packages...
success Saved lockfile.
Done in 0.38s.
/ # node -v
v6.10.1

Why I yarn installing and replacing node installed?

[Daniel15]

I think doing apk add nodejs-current yarn will get you 7.x instead of 6.x, but you'll need to ask the maintainer of the Alpine package (see https://pkgs.alpinelinux.org/package/edge/community/x86/yarn)

[mhart]

@whitecolor because node in the base image wasn't installed using apk – so the yarn apk package knows nothing about it

[mhart]

@whitecolor you should install yarn manually if you want to use that docker image – see these comments: https://github.com/yarnpkg/yarn/issues/1326#issuecomment-277539561 or https://github.com/yarnpkg/yarn/issues/1326#issuecomment-277558635

[ide]

Thanks to @mhart, Yarn is now included in the alpine-node (v8+) Docker image: https://github.com/mhart/alpine-node/issues/65#issuecomment-305056795. This is especially useful since we've found some issues with npm 5 so having an alternative out of the box is great.

[whitecolor]

Now images should be updated more often) as Yarn and NPM5 still have a lot of minor bugs

[whitecolor]

Isn't there automatic publishing in alipne repo? https://pkgs.alpinelinux.org/packages?name=yarn&branch=&repo=&arch=&maintainer=

The latest available version is still 0.24

[Daniel15]

The Alpine repo is community maintained; it's not maintained by the Yarn team. You'd need to contact its maintainer to get the package updated.

[andregomars]

RUN apk update
RUN apk add yarn

[errm]

Hi I am the maintainer of the alpine package... I have been tracking the stable version of yarn. Feel free to ping me if I miss an update.

The latest release of alpine linux (3.7) includes yarn 1.3.2

Note that unless there is a security issue, the version included in 3.7 won't be updated... to get newer versions (when they become stable) you will need to wait for a new release of alpine (roughly every 6 months) or use the edge version (not recommended for a production system).

[deadhexagon]

For those coming here from search results and are looking to install yarn from Alpine repos, it is indeed included in v3.7. Since it wasn't mentioned here, you need to enable the community repo in /etc/apk/repositories. Just uncomment that line, then run: apk update and apk add yarn as root.

NodeJS is already available in the main repo and is v8.9.3, so no worries there.

Thank you @errm for maintaining this.

[damien-roche]

Having a lot of fun between this and Webpack 4! :P