Sharp: Cannot install sharp on ARM64 Odroid Ubuntu

Created on 17 Apr 2017 · 24Comments · Source: lovell/sharp

Hi ,

I've been trying to install sharp on Ubuntu 16.04 Xenial installed on an Odroid but am not able to get the dependency libvips installed on it. Which obviously is causing installation of sharp to fail.

I'm running node v7.9.x and npm is v4.2.0.

Here is the output when I try to install sharp "_sudo npm install sharp_"

[email protected] install /home/pankaj/node_modules/sharp
node-gyp rebuild

ERROR: Download of https://dl.bintray.com/lovell/sharp/libvips-8.4.2-linux-arm64.tar.gz failed: Response code 404 (Not Found)
gyp: Call to 'LDD_VERSION="ldd (Ubuntu GLIBC 2.23-0ubuntu3) 2.23
Copyright (C) 2016 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Written by Roland McGrath and Ulrich Drepper." node -e "require('./binding').download_vips()"' returned exit status 1 while in binding.gyp. while trying to load binding.gyp
gyp ERR! configure error
gyp ERR! stack Error: gyp failed with exit code: 1
gyp ERR! stack at ChildProcess.onCpExit (/usr/lib/node_modules/npm/node_modules/node-gyp/lib/configure.js:308:16)
gyp ERR! stack at emitTwo (events.js:106:13)
gyp ERR! stack at ChildProcess.emit (events.js:194:7)
gyp ERR! stack at Process.ChildProcess._handle.onexit (internal/child_process.js:215:12)
gyp ERR! System Linux 3.14.29
gyp ERR! command "/usr/bin/nodejs" "/usr/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
gyp ERR! cwd /home/pankaj/node_modules/sharp
gyp ERR! node -v v7.9.0
gyp ERR! node-gyp -v v3.5.0
gyp ERR! not ok
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@^1.0.0 (node_modules/chokidar/node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for [email protected]: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"arm64"})
npm WARN enoent ENOENT: no such file or directory, open '/home/pankaj/package.json'
npm WARN pankaj No description
npm WARN pankaj No repository field.
npm WARN pankaj No README data
npm WARN pankaj No license field.
npm ERR! Linux 3.14.29
npm ERR! argv "/usr/bin/nodejs" "/usr/bin/npm" "install" "sharp"
npm ERR! node v7.9.0
npm ERR! npm v4.2.0
npm ERR! code ELIFECYCLE
npm ERR! errno 1

npm ERR! [email protected] install: node-gyp rebuild
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] install script 'node-gyp rebuild'.
npm ERR! Make sure you have the latest version of node.js and npm installed.
npm ERR! If you do, this is most likely a problem with the sharp package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR! node-gyp rebuild
npm ERR! You can get information on how to open an issue for this project with:
npm ERR! npm bugs sharp
npm ERR! Or if that isn't available, you can get their info via:
npm ERR! npm owner ls sharp
npm ERR! There is likely additional logging output above.

npm ERR! Please include the following file with any support request:
npm ERR! /home/pankaj/.npm/_logs/2017-04-17T08_03_38_451Z-debug.log

I followed the steps on sharp.dimens.io:

Added repository: _sudo add-apt-repository -y ppa:lovell/trusty-backport-vips_
gpg: keyring /tmp/tmp33esnaad/secring.gpg' created gpg: keyring/tmp/tmp33esnaad/pubring.gpg' created
gpg: requesting key 21794330 from hkp server keyserver.ubuntu.com
gpg: /tmp/tmp33esnaad/trustdb.gpg: trustdb created
gpg: key 21794330: public key "Launchpad PPA for Lovell Fuller" imported
gpg: Total number processed: 1
gpg: imported: 1 (RSA: 1)
OK

then did - _sudo apt-get update_
Here I get the following error in output:

My sources contains: _/etc/apt/sources.list_

deb http://ports.ubuntu.com/ubuntu-ports/ xenial main universe restricted multiverse
deb-src http://ports.ubuntu.com/ubuntu-ports/ xenial main universe restricted multiverse
deb http://ports.ubuntu.com/ubuntu-ports/ xenial-updates main universe restricted multiverse
deb-src http://ports.ubuntu.com/ubuntu-ports/ xenial-updates main universe restricted multiverse
deb http://ports.ubuntu.com/ubuntu-ports/ xenial-backports main restricted universe multiverse
deb-src http://ports.ubuntu.com/ubuntu-ports/ xenial-backports main restricted universe multiverse

deb http://ports.ubuntu.com/ubuntu-ports/ xenial-security main restricted universe multiverse
deb-src http://ports.ubuntu.com/ubuntu-ports/ xenial-security main restricted universe multiverse

deb http://ports.ubuntu.com/ubuntu-ports/ xenial-proposed main restricted universe multiverse
deb-src http://ports.ubuntu.com/ubuntu-ports/ xenial-proposed main restricted universe multiverse

uname -a
Linux odroid64 3.14.29 #25 SMP PREEMPT Fri Apr 22 16:24:15 EDT 2016 aarch64 aarch64 aarch64 GNU/Linux

I've spent hours searching and trying different things but none have worked. Please advice how to fix this. Thanks in advance!

bug

Source

ajiratech2

Most helpful comment

Great, thanks for confirming. 64-bit ARM (ARMv8) support will be fixed properly in the v0.18.0 release. (I'm also expanding the ARM packaging tests to help prevent further regression.)

lovell on 19 Apr 2017

❤1 🎉1

All 24 comments

Hello, it looks like the value for node's process.arch attribute on 64-bit ARM chips has now settled on (the undocumented) "arm64" rather than "arch64" as currently checked (which I think should have been "aarch64", so that check is probably wrong anyway).

Are you able to make the following change from arch64 to arm64 in binding.js#L45-L46 and see if that works?

-  if (arch === 'arm' || arch === 'armhf' || arch === 'arch64') {
-    const armVersion = (arch === 'arch64') ? '8' : process.env.npm_config_armv || process.config.variables.arm_version || '6';
+  if (arch === 'arm' || arch === 'armhf' || arch === 'arm64') {
+    const armVersion = (arch === 'arm64') ? '8' : process.env.npm_config_armv ||  process.config.variables.arm_version || '6';

lovell on 17 Apr 2017

I searched on my installation and did not find the file "binding.js" anywhere on it. Since sharp is not yet installed I do not have "node_modules/sharp" created yet.

Are you suggesting I download sharp code into my "node_modules" folder and then modify this file and try again ?

ajiratech2 on 17 Apr 2017

I cloned the repo into my "node_modules" directory and modified the file "binding.js" with the changes you'd suggested. Then I tried to install sharp again and here is what I get :

pankaj@odroid64:~$ sudo npm install sharp
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@^1.0.0 (node_modules/chokidar/node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for [email protected]: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"arm64"})
npm WARN enoent ENOENT: no such file or directory, open '/home/pankaj/package.json'
npm ERR! Linux 3.14.29
npm ERR! argv "/usr/bin/nodejs" "/usr/bin/npm" "install" "sharp"
npm ERR! node v7.9.0
npm ERR! npm v4.2.0
npm ERR! path /home/pankaj/node_modules/sharp
npm ERR! code EISGIT

npm ERR! git /home/pankaj/node_modules/sharp: Appears to be a git repo or submodule.
npm ERR! git /home/pankaj/node_modules/sharp
npm ERR! git Refusing to remove it. Update manually,
npm ERR! git or move it out of the way first.

npm ERR! Please include the following file with any support request:
npm ERR! /home/pankaj/.npm/_logs/2017-04-17T18_40_04_505Z-debug.log

ajiratech2 on 17 Apr 2017

I've made the change on a temporary branch. Please can you test this "version" in your package.json by placing a dependency on it using:

  "sharp": "lovell/sharp#arm64"

lovell on 17 Apr 2017

Thank you !! The install successfully completed. I'll test it out in another few.

How do I go about for future such devices ?

pankaj@odroid64:~$ sudo npm install sharp

[email protected] install /home/pankaj/node_modules/sharp
node-gyp rebuild

make: Entering directory '/home/pankaj/node_modules/sharp/build'
TOUCH Release/obj.target/libvips-cpp.stamp
CXX(target) Release/obj.target/sharp/src/common.o
CXX(target) Release/obj.target/sharp/src/metadata.o
CXX(target) Release/obj.target/sharp/src/operations.o
CXX(target) Release/obj.target/sharp/src/pipeline.o
CXX(target) Release/obj.target/sharp/src/sharp.o
CXX(target) Release/obj.target/sharp/src/utilities.o
SOLINK_MODULE(target) Release/obj.target/sharp.node
COPY Release/sharp.node
TOUCH Release/obj.target/win_copy_dlls.stamp
make: Leaving directory '/home/pankaj/node_modules/sharp/build'
[email protected] /home/pankaj
└─┬ [email protected] (git://github.com/lovell/sharp.git#fc1a3562c3d6b3f9cab2554bbed9c6dfacdbd03f)
├─┬ [email protected]
│ ├─┬ [email protected]
│ │ └─┬ [email protected]
│ │ ├── [email protected]
│ │ ├── [email protected]
│ │ └── [email protected]
│ └── [email protected]
├─┬ [email protected]
│ ├─┬ [email protected]
│ │ └── [email protected]
│ └─┬ [email protected]
│ └─┬ [email protected]
│ └── [email protected]
└─┬ [email protected]
├─┬ [email protected]
│ └── [email protected]
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected]
└─┬ [email protected]
└── [email protected]

npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@^1.0.0 (node_modules/chokidar/node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for [email protected]: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"arm64"})

ajiratech2 on 17 Apr 2017

I've completed setting up the environment and during testing of sharp/libvips I get a warning that is supposed to be harmless and after that the node application crashes. Here is the output:

------ error from my application -------

INFO: Entered 'POST/downloadImage'
vips warning: VipsJpeg: error reading resolution
*** Error in `node': free(): invalid size: 0x0000007f60001230 ***
Aborted

------- end of error message, application crashed --------

Here is the line of code just prior to the log message "INFO: Entered POST/downloadImage":

      sharp(fileToDownload)
         .resize(maxSize, maxSize)
         .toBuffer(function(err, outputBuffer) {
           if (err) {
              logger.log(logger.levelInfo, "sharp.read -error occurred : " + err);
           }
           else{

              res.writeHead(200, {'Content-Type': 'image/jpeg' });
              res.end( outputBuffer );
           }
      });

Sharp version that got installed after the workaround you put in place - 0.17.3
libvips42/xenial,now 8.2.2-1 arm64 [installed]

The same code worked when node was running on an X86 or even another ARM board (Allwinner). Please advice how to resolve this?

ajiratech2 on 18 Apr 2017

I came across issue # 567
It has similar error message:

*** Error in `node': free(): invalid size: 0x0000007f60001230 ***
Aborted

Your advice there was to install libvips via npm and not apt-get, so I uninstalled libvips-dev and libgsf-1-dev and then installed libvips using npm successfully.

However now that same point in code aborts with the following error message:

INFO: Entered 'POST/downloadImage'
node: malloc.c:2928: __libc_malloc: Assertion `!victim || chunk_is_mmapped (mem2chunk (victim)) || ar_ptr == arena_for_chunk (mem2chunk (victim))' failed.
Aborted

Please advice.

ajiratech2 on 18 Apr 2017

Glad to hear the "arm64" fix allowed you to install - I'll add this change to the v0.18.0 release.

malloc.c:2928: __libc_malloc: Assertion `!victim || chunk_is_mmapped (mem2chunk (victim)) || ar_ptr == arena_for_chunk (mem2chunk (victim))' failed.

This looks like a memory allocation failure, or possibly memory corruption. Are you able to share the smallest possible code sample that fails in this way?

lovell on 18 Apr 2017

sure. Here is the code that gets called and crashes the app:

ImageSizeOriginal is set to very high value - 999999999

posted the correct actual code. This error was tested and experienced in POST, so providing that code.


app.post('/downloadImage', function(req, res) {

    logger.log(logger.levelInfo,"Entered 'POST/downloadImage'" );

    var maxSize = req.body.maxSize;
                              // req.body.pathToFile includes <device name>/<subFolder>/
    var absolutePathToFile = mediaParentFolder + req.body.pathToFile + req.body.filename;

    logger.log(logger.levelDebug, "POST/downloadImage: will try to read file at location : " + absolutePathToFile);

    if(maxSize < ImageSizeOriginal){
      logger.log(logger.levelDebug, "will send reduced size of " + maxSize + " from file at location : " + absolutePathToFile);

      sharp(absolutePathToFile)
         .resize(maxSize, maxSize)
         .toBuffer(function(err, outputBuffer) {

           if (err) {
              logger.log(logger.levelInfo, "sharp.read -error occurred : " + err);
           }
           else{

              logger.log(logger.levelInfo, "Sharp optimization successful");
              res.writeHead(200, {'Content-Type': 'image/jpeg' });
              res.end( outputBuffer );
              logger.log(logger.levelDebug, "sent bytes : " + outputBuffer.length);
           }
      });

    }
    else {  // I think this will even catch maxsize as 'undefined'

      logger.log(logger.levelDebug, "will send Original size from file at location : " + absolutePathToFile);

      fs.readFile(absolutePathToFile, function read(err, data) {

         if (err) {
               logger.log(logger.levelInfo, "error occurred : " + err);
               logger.log(logger.levelInfo, "data holds : '" + data + "'");
         }

         res.writeHead(200, {'Content-Type': 'image/jpeg' });
         res.end(data);

        logger.log(logger.levelDebug, "sent bytes : " + data.length);

      });

    }

});

Here is the error it generates:

INFO: Entered 'POST/downloadImage'
INFO: Entered 'POST/downloadImage'
vips warning: VipsJpeg: error reading resolution
*** Error in `node': corrupted double-linked list: 0x0000007f78081e60 ***
Aborted

Please let me know if you'd like to see additional code. Thanks

ajiratech2 on 18 Apr 2017

Are you able to reduce this example further, without all the HTTP stuff? Ideally this will be self-contained code that can be run at the command line yet still produce the error.

lovell on 18 Apr 2017

Yes. I've done that now and here it is:

var sharp = require("sharp");

      sharp("DDD89B0C-FB3C-4F32-A4FD-186CE6C6D290.jpg")
         .resize(150, 150)
         .toBuffer(function(err, outputBuffer) {

           if (err) {
              logger.log(logger.levelInfo, "sharp.read -error occurred : " + err);
           }
           else{

              console.log("Sharp optimization successful");
              console.log("sending bytes : " + outputBuffer.length);
           }
      });

Here is the complete set of logs and error it generates:

pankaj@odroid64:~/test$ node testsharp.js 
vips warning: VipsJpeg: error reading resolution
*** Error in `node': free(): invalid size: 0x0000007f98000a60 ***
Aborted

ajiratech2 on 18 Apr 2017

Great, thank you, that's a very straightforward example so hopefully I'll be able to reproduce/confirm this. (But probably won't get around to this for a few days though.)

lovell on 18 Apr 2017

Sure, no worries.

I appreciate your time and effort in resolving this.

I’m going to continue trying a few things to see if I can provide more information on this. Also, I’m working on making it accessible via SSH so you can access this installation from your end (even if you break it, it’ll be OK). Once it’s available over internet, I’ll let you know.

If there is anything I can do from my end then let me know.

On Apr 18, 2017, at 5:25 PM, Lovell Fuller <[email protected]notifications@github.com> wrote:

Great, thank you, that's a very straightforward example so hopefully I'll be able to reproduce/confirm this. (But probably won't get around to this for a few days though.)

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://github.com/lovell/sharp/issues/772#issuecomment-294987352, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ANIxL3CF8argpjsF2E2Yw1CEsox2nY_Eks5rxSpkgaJpZM4M-9-S.

ajiratech2 on 19 Apr 2017

I tested the same code on another ARM platform and it worked just fine.

uname -a
Linux FriendlyARM 3.4.39-h3 #26 SMP PREEMPT Sat Feb 4 16:59:41 CST 2017 armv7l GNU/Linux

It is only with this S905 ARM that we see the crash.

ajiratech2 on 19 Apr 2017

I added a temporary commit https://github.com/lovell/sharp/commit/88b9f4ba87d72739682de1178337a570fb633f49 on the arm64 branch that I think might help, please can you try installing from it again.

lovell on 19 Apr 2017

Sure. I’ll try it right now and revert shortly.

ajiratech2 on 19 Apr 2017

Awesome. That fixed it.

It works beautifully now. Thank you!! I appreciate your prompt help and fix on this.

ajiratech2 on 19 Apr 2017

Great, thanks for confirming. 64-bit ARM (ARMv8) support will be fixed properly in the v0.18.0 release. (I'm also expanding the ARM packaging tests to help prevent further regression.)

lovell on 19 Apr 2017

❤1 🎉1

Thanks!

So for future installs I can simply install sharp without having to do following in dependencies:

"sharp" : "lovell/sharp#arm64"

Correct?

ajiratech2 on 20 Apr 2017

@ajiratech2 Correct. This issue is tagged as v0.18.0 so you'll get a notification when it is available via npm.

lovell on 20 Apr 2017

v0.18.0 now available, thanks for reporting this.

lovell on 30 May 2017

Thank you!

I appreciate your help in getting this resolved and adding it to mainline.

On May 30, 2017, at 3:41 AM, Lovell Fuller <[email protected]notifications@github.com> wrote:

v0.18.0 now available, thanks for reporting this.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/lovell/sharp/issues/772#issuecomment-304800072, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ANIxL79iRpbU-jwE6Moprt0YZ38YG1FDks5r-8hEgaJpZM4M-9-S.

ajiratech2 on 30 May 2017

Sorry for commenting from a closed issue. But is there any precompiled ARMHF (32 bit) binaries available?

We recently switched to sharp (from external imagemagick / graphicmagik), but our Raspberry Pi snapcraft builds will only pull in the ARM64 binaries - resulting in wrong ELF class broken builds 😞 Thanks.