Sharp: Build fails in Alpine Linux

[download13]

When trying to npm install sharp under Alpine it fails with the following error:

> node-gyp rebuild                                                                    

/root/node_modules/semver/semver.js:293                                               
    throw new TypeError('Invalid Version: ' + version);                               
    ^                                                                                 

TypeError: Invalid Version: libc.0                                                    
    at new SemVer (/root/node_modules/semver/semver.js:293:11)                        
    at compare (/root/node_modules/semver/semver.js:566:10)                           
    at Function.lt (/root/node_modules/semver/semver.js:600:10)                       
    at Object.module.exports.download_vips (/root/node_modules/sharp/binding.js:68:33)
    at [eval]:1:22                                                                    
    at Object.exports.runInThisContext (vm.js:54:17)                                  
    at Object.<anonymous> ([eval]-wrapper:6:22)                                       
    at Module._compile (module.js:435:26)                                             
    at node.js:578:27                                                                 
    at doNTCallback0 (node.js:419:9)                                                  
gyp: Call to 'LDD_VERSION="musl libc                                                  
Version 1.1.12                                                                        
Dynamic Program Loader                                                                

Alpine uses musl as it's libc
The problem seems to be that the method of detecting the current version of glibc breaks when faced with the musl version string: "musl libc", and probably with any other libc that doesn't maintain version parity with glibc.

Is there any way that the musl version string could be accommodated by the detection code?

Thanks!

[lovell]

Hi Erin, thanks for reporting this. You've probably seen #242 where we got everything working on Alpine previously.

I'll add Alpine to the Docker-based packaging tests to help prevent regressions like this.

[lovell]

The pre-built Linux binaries are glibc-dependent so Alpine/musl users will need to compile libvips from source.

@wjordan Are you still maintaining https://github.com/wjordan/dockerfile-libvips ?

[wjordan]

Just gave dockerfile-libvips a refresh and pushed a new Docker image to wjordan/libvips, so the install-script and/or prebuilt docker image should be a decent starting-point for compiling the latest Vips source on alpine.

I've also converted this build script to a proper Alpine apk package, so using this (currently self-hosted, x86_64 only) apk build should work on Alpine instead of compiling libvips from source:

apk add libvips --repository https://s3.amazonaws.com/wjordan-apk --allow-untrusted

Also added wjordan/docker-sharp (Docker image at wjordan/sharp) with Sharp pre-compiled.

Finally, also take a look at wjordan/dockerfile-image-resizer for an example of how to use the pre-compiled Sharp package in another project (in this case a fork of image-resizer).

[lovell]

Thank you @wjordan, I've used your Docker image as the basis for the new Alpine-based test runner. There are 4 failing test cases at the moment, all related to (a lack of) lcms2. These should pass the next time you push the latest wjordan/libvips to the Docker hub.

I get a segfault when a libvips' cache invalidation occurs, most likely due to the smaller size of the musl stack. Here's an example 640-deep backtrace:

#0  0x00007ffff5eaf190 in ?? () from /usr/lib/libglib-2.0.so.0
#1  0x00007ffff5eaf6b1 in g_hash_table_lookup () from /usr/lib/libglib-2.0.so.0
...
#506 0x00007ffff619d22e in g_object_unref () from /usr/lib/libgobject-2.0.so.0
...
#509 0x00007ffff642b251 in vips_cache_operation_buildp () from /usr/lib/libvips.so.42
...
#516 0x00007ffff6433da3 in vips.jpeg_read_file () from /usr/lib/libvips.so.42
...
#633 0x00007ffff64c3d20 in vips_region_fill () from /usr/lib/libvips.so.42
#634 0x00007ffff64c3dd4 in vips_region_prepare () from /usr/lib/libvips.so.42
...
#638 0x00007ffff5ed644d in ?? () from /usr/lib/libglib-2.0.so.0
#639 0x00007ffff7dc70de in ?? () from /lib/ld-musl-x86_64.so.1

My recommendation would be to disable libvips' cache via sharp.cache(false); when using Alpine/musl.

[lovell]

@wjordan It looks like libvips' pkgconfig is missing from your latest wjordan/libvips image:

$ docker run --rm wjordan/libvips sh -c "ls -al /usr/lib/pkgconfig/vips*"
ls: /usr/lib/pkgconfig/vips*: No such file or directory

The (older) wjordan/sharp image contains these files:

$ docker run --rm wjordan/sharp sh -c "ls -al /usr/lib/pkgconfig/vips*"
-rw-r--r--    1 root     root           232 Feb  9 17:51 /usr/lib/pkgconfig/vips-cpp.pc
-rw-r--r--    1 root     root           349 Feb  9 17:51 /usr/lib/pkgconfig/vips.pc
-rw-r--r--    1 root     root           234 Feb  9 17:51 /usr/lib/pkgconfig/vipsCC.pc

Any ideas? (Asking here as the wjordan/libvips repo has no "issues".)

[wjordan]

Ah sorry- learning the various packaging conventions as I go here. Submitted #359 which is my attempt at outlining a more maintainable approach for test-packaging Sharp on alpine linux by using OS packages instead of Docker images.

[lovell]

v0.13.0 is now available and tested against Alpine Linux, thanks both for your time.

[raphaelbs]

Hello,
I'm trying to build a docker image with sharp but it fails to the same error.
First I tried to build using the alpine:8 and the libvips (suggested by @lovell) that is present in the alpine repository.

My Dockerfile:

Libvips was successfully installed:

But when sharp tries to compile:

Then I saw the @wjordan docker-sharp and follow his instructions to build the image with sharp installed globally into the alpine edge image.

All alpine packages installed successfully:

But unfortunately I got the same error:

As I am new to Docker I know that I'm missing something. I appreciate any help.
Thank you guys, you made incredible tools!

[sabrehagen]

This is how I install my vips dependency

RUN apk add --update \
    --repository http://dl-3.alpinelinux.org/alpine/edge/testing \
    vips-tools vips-dev fftw-dev \
    && rm -rf /var/cache/apk/*

[raphaelbs]

@sabrehagen got me in the right direction.
For future reference that is how I got it working.

Dockerfile:

RUN apk add --update --no-cache \
  --repository http://dl-3.alpinelinux.org/alpine/edge/testing \
  vips-dev fftw-dev gcc g++ make libc6-compat \

Without libc6-compat the docker image builds successfully but when the server starts it throws:
_Error loading shared library ld-linux-x86-64.so.2_

Sharp version 0.18.2
Node alpine: node:8-alpine, wich uses alpine:3.6
83 Dependencies installed in Alpine:

1. busybox-1.26.2-r5
2. alpine-baselayout-3.0.4-r0
3. alpine-keys-2.1-r1
4. libressl2.5-libcrypto-2.5.4-r0
5. libressl2.5-libssl-2.5.4-r0
6. zlib-1.2.11-r0
7. apk-tools-2.7.2-r0
8. scanelf-1.2.2-r0
9. libc-utils-0.7.1-r0
10. libgcc-6.3.0-r4
11. libstdc++-6.3.0-r4
12. musl-1.1.16-r13
13. musl-utils-1.1.16-r13
14. fftw-double-libs-3.3.6p1-r0
15. fftw-long-double-libs-3.3.6p1-r0
16. fftw-single-libs-3.3.6p1-r0
17. pkgconf-1.3.7-r0
18. fftw-dev-3.3.6p1-r0
19. binutils-libs-2.28-r2
20. binutils-2.28-r2
21. gmp-6.1.2-r0
22. isl-0.17.1-r0
23. libgomp-6.3.0-r4
24. libatomic-6.3.0-r4
25. mpfr3-3.1.5-r0
26. mpc1-1.0.3-r0
27. gcc-6.3.0-r4
28. musl-dev-1.1.16-r13
29. libc-dev-0.7.1-r0
30. g++-6.3.0-r4
31. libc6-compat-1.1.16-r13
32. make-4.2.1-r0
33. libjpeg-turbo-1.5.1-r0
34. libjpeg-turbo-dev-1.5.1-r0
35. libexif-0.6.21-r1
36. libexif-dev-0.6.21-r1
37. zlib-dev-1.2.11-r0
38. tiff-4.0.7-r3
39. tiff-dev-4.0.7-r3
40. lcms2-2.8-r1
41. lcms2-dev-2.8-r1
42. perl-5.24.1-r2
43. libbz2-1.0.6-r5
44. expat-2.2.0-r1
45. libffi-3.2.1-r3
46. gdbm-1.12-r0
47. ncurses-terminfo-base-6.0-r7
48. ncurses-terminfo-6.0-r7
49. ncurses-libs-6.0-r7
50. readline-6.3.008-r5
51. sqlite-libs-3.18.0-r0
52. python2-2.7.13-r1
53. gettext-asprintf-0.19.8.1-r1
54. libintl-0.19.8.1-r1
55. libunistring-0.9.7-r0
56. gettext-libs-0.19.8.1-r1
57. libxml2-2.9.4-r4
58. gettext-0.19.8.1-r1
59. gettext-dev-0.19.8.1-r1
60. bzip2-dev-1.0.6-r5
61. libffi-dev-3.2.1-r3
62. libuuid-2.28.2-r2
63. libblkid-2.28.2-r2
64. libfdisk-2.28.2-r2
65. libmount-2.28.2-r2
66. libsmartcols-2.28.2-r2
67. util-linux-dev-2.28.2-r2
68. pcre-8.40-r2
69. glib-2.52.1-r0
70. libpcre16-8.40-r2
71. libpcre32-8.40-r2
72. libpcrecpp-8.40-r2
73. pcre-dev-8.40-r2
74. glib-dev-2.52.1-r0
75. libpng-1.6.29-r1
76. libpng-dev-1.6.29-r1
77. libwebp-0.6.0-r0
78. libwebp-dev-0.6.0-r0
79. orc-0.4.26-r0
80. orc-dev-0.4.26-r0
81. giflib-5.1.4-r1
82. vips-8.5.6-r1
83. vips-dev-8.5.6-r1

Thank you all. Keep up with the good work.
Cheers from Brazil.

[sabrehagen]

Thank you for your work @raphaelbs. I struggled with the Error loading shared library ld-linux-x86-64.so.2 myself but never resolved it. Instead I build Node.js from source and leave the build tools and source in the container. This resolved my shared library error, but I did not understand why.

Maybe libc6-compat was all I needed. I will try it and report back.

[raphaelbs]

After some try and error I could minimize a little bit the final Docker image (from 480MB to 317Mb) by removing some dependencies from Alpine.

FROM node:8-alpine

# Define WORKDIR
WORKDIR /usr/src/app
# Copia apenas package.json
COPY package.json /usr/src/app/
# Instala todas as dependencias
RUN apk add --update --repository http://dl-3.alpinelinux.org/alpine/edge/testing \
    vips-dev fftw-dev gcc g++ make libc6-compat && \
    mkdir -p /usr/src/app && \
    npm install --production && \
    apk del gcc g++ make fftw-dev && \
    rm -rf /var/cache/apk/*

# Habilita portas
EXPOSE 80 443

# Copia os arquivos do servidor
COPY . /usr/src/app/

CMD ["npm", "run", "docker-start"]

Here is a list of the remaining 67 apk's in Alpine:

1. alpine-baselayout-3.0.4-r0
2. alpine-keys-2.1-r1
3. libressl2.5-libcrypto-2.5.4-r0
4. libressl2.5-libssl-2.5.4-r0
5. zlib-1.2.11-r0
6. apk-tools-2.7.2-r0
7. scanelf-1.2.2-r0
8. libc-utils-0.7.1-r0
9. libgcc-6.3.0-r4
10. libstdc++-6.3.0-r4
11. musl-1.1.16-r13
12. musl-utils-1.1.16-r13
13. fftw-double-libs-3.3.6p1-r0
14. pkgconf-1.3.7-r0
15. libgomp-6.3.0-r4
16. libc6-compat-1.1.16-r13
17. libjpeg-turbo-1.5.1-r0
18. libjpeg-turbo-dev-1.5.1-r0
19. libexif-0.6.21-r1
20. libexif-dev-0.6.21-r1
21. zlib-dev-1.2.11-r0
22. tiff-4.0.7-r3
23. tiff-dev-4.0.7-r3
24. lcms2-2.8-r1
25. lcms2-dev-2.8-r1
26. perl-5.24.1-r2
27. libbz2-1.0.6-r5
28. expat-2.2.0-r1
29. libffi-3.2.1-r3
30. gdbm-1.12-r0
31. ncurses-terminfo-base-6.0-r7
32. ncurses-terminfo-6.0-r7
33. ncurses-libs-6.0-r7
34. readline-6.3.008-r5
35. sqlite-libs-3.18.0-r0
36. python2-2.7.13-r1
37. gettext-asprintf-0.19.8.1-r1
38. libintl-0.19.8.1-r1
39. libunistring-0.9.7-r0
40. gettext-libs-0.19.8.1-r1
41. libxml2-2.9.4-r4
42. gettext-0.19.8.1-r1
43. gettext-dev-0.19.8.1-r1
44. bzip2-dev-1.0.6-r5
45. libffi-dev-3.2.1-r3
46. libuuid-2.28.2-r2
47. libblkid-2.28.2-r2
48. libfdisk-2.28.2-r2
49. libmount-2.28.2-r2
50. libsmartcols-2.28.2-r2
51. util-linux-dev-2.28.2-r2
52. pcre-8.40-r2
53. glib-2.52.1-r0
54. libpcre16-8.40-r2
55. libpcre32-8.40-r2
56. libpcrecpp-8.40-r2
57. pcre-dev-8.40-r2
58. glib-dev-2.52.1-r0
59. libpng-1.6.29-r1
60. libpng-dev-1.6.29-r1
61. libwebp-0.6.0-r0
62. libwebp-dev-0.6.0-r0
63. orc-0.4.26-r0
64. orc-dev-0.4.26-r0
65. giflib-5.1.4-r1
66. vips-8.5.6-r1
67. vips-dev-8.5.6-r1

There's any chance to reduce even more? I know that I'll not have to upload the same image over and over but would be nice to get the minimum possible. Thanks!

[lovell]

@raphaelbs In terms of disk space, once everything is built, you probably won't need the *-dev packages. apk has a --virtual flag that can help with this.