Sharp: Heroku 64-bit Ubuntu 14.04 + yarn: Cannot find module '../build/Release/sharp.node'

[samjbmason]

When attempting to build a project on heroku we are experiencing an issue where sharp.node is not being found. Below is the heroku build output:

Downloading and installing node 6.9.1...
       Using default npm version: 3.10.8
       Downloading and installing yarn (0.20.3)...
       Installed yarn 0.20.3
-----> Restoring cache
       Loading 2 from cacheDirectories (default):
       - node_modules
       - bower_components (not cached - skipping)
-----> Building dependencies
       Installing node modules (yarn.lock)
       yarn install v0.20.3
       [1/4] Resolving packages...
       [2/4] Fetching packages...
       warning [email protected]: The platform "linux" is incompatible with this module.
       info "[email protected]" is an optional dependency and failed compatibility check. Excluding it from installation.
       [3/4] Linking dependencies...
       warning "[email protected]" has incorrect peer dependency "react-redux@^3.0.0 || ^4.0.0".
       [4/4] Building fresh packages...
       $ gulp build
       module.js:471
       throw err;
       ^

       Error: Cannot find module '../build/Release/sharp.node'
       at Function.Module._resolveFilename (module.js:469:15)
       at Function.Module._load (module.js:417:25)
       at Module.require (module.js:497:17)
       at require (internal/module.js:20:19)
       at Object.<anonymous> (/tmp/build_a0cdb656013074c78176c1bdebea3455/Lostmyname-website-d9b76f1e1644504c47a0c934bed5fd92db728cb1/node_modules/sharp/lib/constructor.js:8:15)
       at Module._compile (module.js:570:32)
       at Object.Module._extensions..js (module.js:579:10)
       at Module.load (module.js:487:32)
       at tryModuleLoad (module.js:446:12)
       at Function.Module._load (module.js:438:3)
       error Command failed with exit code 1.
       info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
-----> Build failed

[lovell]

Hello, are you only able to reproduce this on Heroku? If so, it's probably worth you asking them if they can/will support adding the --verbose flag for yarn install.

(It would be a great addition to yarn if it were to support the NPM_CONFIG_LOGLEVEL=verbose environment variable.)

[aaronbuchanan]

Also having this issue, anyone figure out the cause?

[wprater]

happening to us as well. we tried to lock the yarn version to 0.20.3 but once a new yarn.lock was pushed up, it seemed to fail.

[lovell]

We'll need more verbose install-time information from Heroku to help debug this problem. If anyone can make that happen, e.g. via paid support, please do so :)

[griffinpp]

We were experiencing this issue, and got things working again by setting NODE_MODULES_CACHE to false (heroku -a <appname> config:set NODE_MODULES_CACHE=false).

Our best guess, based on some rudimentary testing, is that heroku is not able or not configured to cache binary files. So if caching is enabled, an initial deploy works fine, but a second deploy will attempt to use the cached modules, which do not contain the necessary binary files. Disabling caching forces the application to always use the correct, uncached node_modules directory.

[griffinpp]

Also, in case the devs are still curious, here is the output from running npm install sharp directly with the heroku toolbelt (heroku run -a <appname> 'npm install sharp') while we were still experiencing the problem:

> [email protected] install /app/node_modules/sharp
> node-gyp rebuild

make: Entering directory `/app/node_modules/sharp/build'
  TOUCH Release/obj.target/libvips-cpp.stamp
  CXX(target) Release/obj.target/sharp/src/common.o
  CXX(target) Release/obj.target/sharp/src/metadata.o
  CXX(target) Release/obj.target/sharp/src/operations.o
  CXX(target) Release/obj.target/sharp/src/pipeline.o
  CXX(target) Release/obj.target/sharp/src/sharp.o
  CXX(target) Release/obj.target/sharp/src/utilities.o
  SOLINK_MODULE(target) Release/obj.target/sharp.node
  COPY Release/sharp.node
  TOUCH Release/obj.target/win_copy_dlls.stamp
make: Leaving directory `/app/node_modules/sharp/build'
[email protected] /app
`-- [email protected]
  +-- [email protected]
  | `-- [email protected]
  +-- [email protected]
  | +-- [email protected]
  | | `-- [email protected]
  | `-- [email protected]
  |   `-- [email protected]
  |     `-- [email protected]
  +-- [email protected]
  | +-- [email protected]
  | | `-- [email protected]
  | +-- [email protected]
  | +-- [email protected]
  | +-- [email protected]
  | +-- [email protected]
  | +-- [email protected]
  | +-- [email protected]
  | +-- [email protected]
  | +-- [email protected]
  | +-- [email protected]
  | `-- [email protected]
  |   `-- [email protected]
  `-- [email protected]

npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@^1.0.0 (node_modules/chokidar/node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for [email protected]: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})

This did not solve the problem, as the file (in fact, the entire /build directory) was not present even after the install

[samjbmason]

Yeah we came to the same conclusion, I think due to Heroku's ephemeral file system, while the cache is created the first time the second time when it looks for it the symlink is missing or changed since it's technically a new container

[lovell]

Great detective work all. A PR to update the Heroku docs with a link to the following would be very welcome.

https://devcenter.heroku.com/articles/nodejs-support#cache-behavior

[lovell]

Docs updated in 5e015cc, thanks all.