Sharp: Can't compile under Apple Silicon M1 arm64

[gpetrov]

Did you see the documentation relating to installation?

• yes

Have you ensured the architecture and platform of Node.js used for npm install is the same as the architecture and platform of Node.js used at runtime?

• yes

Are you using the latest version? Is the version currently in use as reported by npm ls sharp the same as the latest version as reported by npm view sharp dist-tags.latest?

• yes

What is the complete output of running npm install --verbose sharp? Have you checked this output for useful error messages?

> [email protected] install /..../node_modules/sharp
> (node install/libvips && node install/dll-copy && prebuild-install) || (node-gyp rebuild && node install/dll-copy)

info sharp Downloading https://github.com/lovell/sharp-libvips/releases/download/v8.10.0/libvips-8.10.0-darwin-arm64v8.tar.br
ERR! sharp Prebuilt libvips 8.10.0 binaries are not yet available for darwin-arm64v8
info sharp Attempting to build from source via node-gyp but this may fail due to the above error
info sharp Please see https://sharp.pixelplumbing.com/install for required dependencies
gyp info it worked if it ends with ok
....
 -gdwarf-2 -mmacosx-version-min=11 -arch arm64 -Wall -Wendif-labels -W -Wno-unused-parameter -std=c++14 -stdlib=libc++ -fexceptions -Wall -O3 -std=c++14 -stdlib=libc++ -MMD -MF ./Release/.deps/Release/obj.target/sharp/src/common.o.d.raw   -c -o Release/obj.target/sharp/src/common.o ../src/common.cc
../src/common.cc:24:10: fatal error: 'vips/vips8' file not found
#include <vips/vips8>
         ^~~~~~~~~~~~
1 error generated.
make: *** [Release/obj.target/sharp/src/common.o] Error 1
gyp ERR! build error 
gyp ERR! stack Error: `make` failed with exit code: 2
gyp ERR! stack     at ChildProcess.onExit (/usr/local/lib/node_modules/nw-gyp/lib/build.js:261:23)
gyp ERR! stack     at ChildProcess.emit (events.js:315:20)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:277:12)
gyp ERR! System Darwin 20.1.0
gyp ERR! command "/usr/local/bin/node" "/usr/local/bin/nw-gyp" "rebuild"

gyp ERR! node -v v14.15.1
gyp ERR! nw-gyp -v v3.6.2
gyp ERR! not ok 

Seems somehow libvips can't be compiled from source

Any ideas?

[lovell]

info sharp Downloading https://github.com/lovell/sharp-libvips/releases/download/v8.10.0/libvips-8.10.0-darwin-arm64v8.tar.br
ERR! sharp Prebuilt libvips 8.10.0 binaries are not yet available for darwin-arm64v8
info sharp Attempting to build from source via node-gyp but this may fail due to the above error

https://sharp.pixelplumbing.com/install#prebuilt-binaries

You'll need to compile libvips from source and install it globally first.

https://sharp.pixelplumbing.com/install#custom-libvips

[gpetrov]

I was already using --build-from-source all the time and even tried SHARP_IGNORE_GLOBAL_LIBVIPS
but with the same results unfortunately.

Should I try to compile libvips completely independent from npm as explained here?
https://libvips.github.io/libvips/install.html#building-libvips-from-a-source-tarball

I thought npm install --build-from-source should take care of that ...

[lovell]

Building libvips from source is independent of building sharp from source, so yes, you'll need to follow the libvips-provided instructions.

We won't be able to provide any prebuilt binaries for Apple Silicon until a CI provider makes ARM64 macOS hardware available for free to open source.

[gpetrov]

Unfortunately homebrew is not yet fully compatible with apple silicon as of https://github.com/Homebrew/brew/issues/7857
so commands like configure and autoconf aren't there yet - which makes difficult to compile libvips

if you have other ideas - I will be glad to try them out on the M1. I was able to compile many other node modules without any problem on Apple Silicon arm64, it's just the libvips that gives me an issue and stops me from completing the last step with the great node module sharp.

Can I just copy the source from libvips so it gets picket up by the npm install --build-from-source ?
As this is just the whole error:

../src/common.cc:24:10: fatal error: 'vips/vips8' file not found

[gpetrov]

Actually I got bit further in compiling libvips under M1 arm64 but unfortunately got stuck in compiling dependencies like libglib2 which are totally unaware of the Mac arm64 and try to use Windows AARM64

So seems there is unfortunately a long way to go to make all this happen. As the guys from homebrew discovered and are working hard to get it all right.

[gpetrov]

Well I finally managed to compile libvips with homebrew under apple arm64, but had to compile it without fftw, libmatio, librsvg and poppler

fftw was requiring gcc as well libmatio, while librsvg needed rust which is not ready yet.
Poppler needed Qt which brought the whole chrome with it, obviously not needed.

trying to build everything statically now, but without luck due to the mentioned bug.

are the prebuild binaries all build statically? Ideally we would like to add a prebuild binary.

[lovell]

The prebuilt binaries for libvips and its (selected) dependencies as provided by sharp are statically linked into a single shared library that can then be dynamically linked to sharp itself. These are created by the scripts in https://github.com/lovell/sharp-libvips/

[medhoover]

I'm interested in this topic as well, did anyone figure a way to get sharp running under Apple silicon m1?

[kenips]

Confirmed from #2467 that this is a non-issue once you have libvips install globally. I wasn't able to install it via brew, but downloaded the tarball and manually compiled fine, with the caveat that I had to manually specify zlib's location: PKG_CONFIG_PATH=/opt/homebrew/Cellar/zlib/1.2.11/lib/pkgconfig ./configure. Once installed globally sharp works fine.

[gpetrov]

@kenips - great to hear you manage to compile it globally! I managed only to compile it with brew, but had to leave a lot of dependencies away. Did you had to leave our some dependencies as well?

Do you have more detailed procedure for the global install? Did you install additional building tools first on a virgin M1, like autoconf, confure and pkgconfig?

[medhoover]

I managed to run sharp thanks to @kenips comment
Here's what I have done:

1. Start with installing Mac ports. You will use ports to install libvips dependencies (follow next step)
2. Install these ports: pkgconfig and glib2. Use this command to install ports: $ sudo port install [port-name]
3. Now depending on the formats you'll be handling, you'll have to install the library for each format. Have look at the libvips dependency list and find them on Mac ports.
   In my case, jpg and png formats are enough, so I installed libjpeg-turbo and libpng

4. Once you've installed the libraries, you're ready to install libvips.

   • Download and unpack the latest libvips release; I used this release.
   • Inside the unpacked folder, run $ ./configure.
   • Then run $ make followed by $ sudo make install

   This should install vips globally. Verify by running: $ which vips

5. When libvips is globally installed, sharp will use the global version. So clean up your node_modules folder and do a fresh install and you're ready to go 👍

[kenips]

@gpetrov yes my procedure is very similar to @medhoover, however using homebrew:

1. Install homebrew arm. I used the steps here.
2. Install pkg-config, zlib normally with brew install
3. Same as above, install any libraries that you need for each image format. I have jpeg, libpng, libtiff, webp installed.
4. Exactly same as @medhoover's steps above, except for configure I had to specify the zlib path with PKG_CONFIG_PATH=/opt/homebrew/Cellar/zlib/1.2.11/lib/pkgconfig ./configure.

I have installed a few packages prior to installing libvips, so there might be additional packages that I've missed for a mint install, so YMMV :).

[abarisain]

Those steps worked for me, thanks! Please note that I also needed to set the PKG_CONFIG_PATH env var when installing node modules.

[shahzore-qureshi]

Thank you @medhoover and @kenips as well as @gpetrov for working on this. Just got myself a Mac Mini M1 and am struggling to get a project running.

Just wanted to add to @medhoover's comment: the port name of pkg-config is pkgconfig. The port install command should be sudo port install pkgconfig.

[junkboy0315]

I was able to get it to work in the following ways with Node.js 15.3.0.
Thanks @medhoover @kenips !

brew install pkg-config glib zlib
brew install libjpeg-turbo libpng webp

# in the libvips source folder
PKG_CONFIG_PATH=/opt/homebrew/Cellar/zlib/1.2.11/lib/pkgconfig ./configure
make
sudo make install

[natew]

I ended up having to run it with to avoid some weird imagemagick/poppler issues (even when installed for arm):

PKG_CONFIG_PATH=/opt/homebrew/Cellar/zlib/1.2.11/lib/pkgconfig ./configure --without-magick --without-poppler

If you're running into issues with x86 libraries, be sure to check your Makefile after you run configure and see if anything is coming from the old homebrew (/usr/local/lib for me). Then install it on arm and re-run the above configure command.

Edit: actually this one failed too, I'm getting this now, even though the directory exists:

../libtool: line 1733: cd: .libs/libvips.lax/libresample.a: No such file or directory

Edit 2: Ended up being because I had a space in the libvips folder name due to downloading two of the same version 😞

Now at:

/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: archive member: .libs/libcolour.a(profile_load.o) cputype (16777223) does not match previous archive members cputype (16777228) (all members must match)

Any reason I've run into so many different issues? Trying a fresh download of libvips to see if perhaps I just messed up something, otherwise will continue mucking around.

Edit:

Deleted and re-downloaded libvips, its saying during configure and then failing now with mismatch arch. Somehow regressed on this..

checking build system type... x86_64-apple-darwin20.2.0

[gpetrov]

🎉 I'm happy to report that thanks to the huge effort of the homebrew team and specially @fxcoudert, libvips can be fully installed on Apple Silicon M1 natively with all its dependencies!

Also for the most of the dependencies prebuild binaries "bottles" are available so installation goes really fast

brew info vips

vips: stable 8.10.5
Image processing library
https://github.com/libvips/libvips
/opt/homebrew/Cellar/vips/8.10.5 (167 files, 14.0MB) *
  Built from source on 2020-12-27 at 16:44:20
From: https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/vips.rb
License: LGPL-2.1-or-later
==> Dependencies
Build: pkg-config ✔
Required: cfitsio ✔, fftw ✔, fontconfig ✔, gettext ✔, giflib ✔, glib ✔, imagemagick ✔, libexif ✔, libgsf ✔, libheif ✔, libimagequant ✔, libmatio ✔, libpng ✔, librsvg ✔, libspng ✔, libtiff ✔, little-cms2 ✔, mozjpeg ✔, openexr ✔, openslide ✔, orc ✔, pango ✔, poppler ✔, webp ✔
==> Analytics
install: 4,198 (30 days), 14,926 (90 days), 42,146 (365 days)
install-on-request: 3,633 (30 days), 12,169 (90 days), 32,332 (365 days)
build-error: 0 (30 days)

brew reinstall vips

==> Downloading https://github.com/libvips/libvips/releases/download/v8.10.5/vips-8.10.5.tar.gz
Already downloaded: /Users/george/Library/Caches/Homebrew/downloads/e8752c98d6977d23909b94c1d3ee31e2c21afda72fa95eeb140bbbbde0add7c8--vips-8.10.5.tar.gz
==> Reinstalling vips 
==> ./configure --prefix=/opt/homebrew/Cellar/vips/8.10.5 --with-magick
==> make install
🍺  /opt/homebrew/Cellar/vips/8.10.5: 167 files, 14.0MB, built in 58 seconds

brew config

HOMEBREW_VERSION: 2.7.0
ORIGIN: https://github.com/Homebrew/brew
HEAD: 993bf95877143afe1cede022083499e0b4fe336c
Last commit: 6 days ago
Core tap ORIGIN: https://github.com/Homebrew/homebrew-core
Core tap HEAD: 27df7ec4599f2aa202586e4d7950afa66e820293
Core tap last commit: 2 hours ago
Core tap branch: master
HOMEBREW_PREFIX: /opt/homebrew
HOMEBREW_CASK_OPTS: []
HOMEBREW_MAKE_JOBS: 8
Homebrew Ruby: 2.6.3 => /System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/bin/ruby
CPU: octa-core 64-bit arm_firestorm_icestorm
Clang: 12.0 build 1200
Git: 2.24.3 => /Applications/Xcode.app/Contents/Developer/usr/bin/git
Curl: 7.64.1 => /usr/bin/curl
macOS: 11.1-arm64
CLT: 12.3.0.0.1.1607026830
Xcode: 12.3
Rosetta 2: false

the same is valid for node and the sharp library now - they can run nicely native on M1 now!