Webpacker: Precompiling assets on Heroku fails with `/bin/sh: 1: npm: not found`

[trcarden]

Summary

When using the latest heroku ruby build pack (https://github.com/heroku/heroku-buildpack-ruby) and deploying a standard rails 5.2+ application the asset compilation yields a broken compilation every time.

Expected Behavior

When you deploy to heroku assets compile with the standard ruby buildpack

Actual behavior

webpacker or related system seems to call out to NPM incorrectly and assets are not able to be compiled.

Repo

Was able to repro by simply deploying a standard empty application with the 4.0.7 release and then tagging master in both package.json and Gemfile. The difference causes a failure.

Output

Failure Output

yarn install v1.16.0
       [1/4] Resolving packages...
       [2/4] Fetching packages...
       info No lockfile found.
       warning package-lock.json found. Your project contains lock files generated by tools other than Yarn. It is advised not to mix package managers in order to avoid resolution inconsistencies caused by unsynchronized lock files. To clear this warning, remove package-lock.json.
       [1/5] Validating package.json...
       [2/5] Resolving packages...
       warning @webpack-contrib/defaults > mrm-core > [email protected]: Please upgrade to kleur@3 or migrate to 'ansi-colors' if you prefer the old syntax. Visit <https://github.com/lukeed/kleur/releases/tag/v3.0.0\> for migration path(s).
       warning @webpack-contrib/defaults > mrm-core > comment-json > [email protected]: `json-parser` is deprecated. Please use `comment-json` instead
       warning [email protected]: This module is no longer maintained, try this instead:
         npm i nyc
       Visit https://istanbul.js.org/integrations for other alternatives.
       warning jest > jest-cli > jest-config > jest-environment-jsdom > jsdom > [email protected]: use String.prototype.padStart()
       [3/5] Fetching packages...
       info [email protected]: The platform "linux" is incompatible with this module.
       info "[email protected]" is an optional dependency and failed compatibility check. Excluding it from installation.
       info [email protected]: The platform "linux" is incompatible with this module.
       info "[email protected]" is an optional dependency and failed compatibility check. Excluding it from installation.
       [4/5] Linking dependencies...
       [5/5] Building fresh packages...
       success Saved lockfile.
       $ npm run build
       /bin/sh: 1: npm: not found
       error Command failed with exit code 127.
       info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.

Output on 4.0.7 tag

       yarn install v1.16.0
       [1/4] Resolving packages...
       [2/4] Fetching packages...
       info No lockfile found.
       warning package-lock.json found. Your project contains lock files generated by tools other than Yarn. It is advised not to mix package managers in order to avoid resolution inconsistencies caused by unsynchronized lock files. To clear this warning, remove package-lock.json.
       [1/5] Validating package.json...
       [2/5] Resolving packages...
       warning @webpack-contrib/defaults > mrm-core > [email protected]: Please upgrade to kleur@3 or migrate to 'ansi-colors' if you prefer the old syntax. Visit <https://github.com/lukeed/kleur/releases/tag/v3.0.0\> for migration path(s).
       warning @webpack-contrib/defaults > mrm-core > comment-json > [email protected]: `json-parser` is deprecated. Please use `comment-json` instead
       warning [email protected]: This module is no longer maintained, try this instead:
         npm i nyc
       Visit https://istanbul.js.org/integrations for other alternatives.
       warning jest > jest-cli > jest-config > jest-environment-jsdom > jsdom > [email protected]: use String.prototype.padStart()
       [3/5] Fetching packages...
       info [email protected]: The platform "linux" is incompatible with this module.
       info "[email protected]" is an optional dependency and failed compatibility check. Excluding it from installation.
       info [email protected]: The platform "linux" is incompatible with this module.
       info "[email protected]" is an optional dependency and failed compatibility check. Excluding it from installation.
       [4/5] Linking dependencies...
       [5/5] Building fresh packages...
       success Saved lockfile.
       info [email protected]: The platform "linux" is incompatible with this module.
       info "[email protected]" is an optional dependency and failed compatibility check. Excluding it from installation.
       warning [email protected]: The engine "browser" appears to be invalid.
       [3/4] Linking dependencies...
       warning " > [email protected]" has unmet peer dependency "@babel/core@^7.0.0".
       [4/4] Building fresh packages...
       Done in 67.42s.
       Compiling…
       Compiled all packs in /tmp/build_f3df98f5306c014d494d7e85b0898b4f/public/packs

[jakeNiemiec]

$ npm run build
       /bin/sh: 1: npm: not found

I think the problem is this line, what is in that script? Is npm installed on the remote?

[hiimtaylorjones]

Hey there,

I would suggest one of two things here:

• Specify an NPM or Yarn engine in your package.json
• Add the Heroku NodeJS buildpack to your app. This will allow your dyno to have npm installed and properly build your app

[trcarden]

Perhaps let me cross link this with another ticket so the maintainer on the heroku buildpack can get involved.

Here is the heroku ruby build pack which should work without having to install the nodejs buildpack (it has yarn and npm support for sprockets and other versions of rails.)

https://github.com/heroku/heroku-buildpack-ruby/issues/912

My hunch was that the latest webpacker system started to rely on npm despite the rails default is still yarn. Given that it works with the webpacker stable branch and fails on master I thought one of you guys would know where that might be coming from. I did a little grepping but it doesn't look like there is a npm reference to build in webpacker.

I originally thought it might be something to do with the node upgrade https://github.com/rails/webpacker/commit/b4057d71e152254819a6f8d1d114c5e374585763 and the way heroku's build pack detects what to do with a rails project.

If I have a bit more time in a week or so I will see if I can debug a bit more and help out

[hiimtaylorjones]

Timothy,

Do you mind providing us with the package.json you're using here? I linked this earlier, but the Ruby buildpack detects whether to use npm or yard from the "engines" section of your package.json.

[adenta]

was this ever resolved? I am running into similar issues described above.

remote: Compressing source files... done.
remote: Building source:
remote:
remote: -----> Ruby app detected
remote: -----> Compiling Ruby/Rails
remote: -----> Using Ruby version: ruby-2.6.3
remote: -----> Installing dependencies using bundler 1.17.3
remote:        Running: bundle install --without development:test --path vendor/bundle --binstubs vendor/bundle/bin -j4 --deployment
remote:        The dependency tzinfo-data (>= 0) will be unused by any of the platforms Bundler is installing for. Bundler is installing for ruby but the dependency is only for x86-mingw32, x86-mswin32, x64-mingw32, java. To add those platforms to the bundle, run `bundle lock --add-platform x86-mingw32 x86-mswin32 x64-mingw32 java`.
remote:        Using rake 13.0.0
remote:        Using concurrent-ruby 1.1.5
remote:        Using i18n 1.7.0
remote:        Using minitest 5.13.0
remote:        Using thread_safe 0.3.6
remote:        Using tzinfo 1.2.5
remote:        Using zeitwerk 2.2.1
remote:        Using activesupport 6.0.1
remote:        Using builder 3.2.3
remote:        Using erubi 1.9.0
remote:        Using mini_portile2 2.4.0
remote:        Using nokogiri 1.10.5
remote:        Using rails-dom-testing 2.0.3
remote:        Using crass 1.0.5
remote:        Using loofah 2.3.1
remote:        Using rails-html-sanitizer 1.3.0
remote:        Using actionview 6.0.1
remote:        Using rack 2.0.7
remote:        Using rack-test 1.1.0
remote:        Using actionpack 6.0.1
remote:        Using nio4r 2.5.2
remote:        Using websocket-extensions 0.1.4
remote:        Using websocket-driver 0.7.1
remote:        Using actioncable 6.0.1
remote:        Using globalid 0.4.2
remote:        Using activejob 6.0.1
remote:        Using activemodel 6.0.1
remote:        Using activerecord 6.0.1
remote:        Using mimemagic 0.3.3
remote:        Using marcel 0.3.3
remote:        Using activestorage 6.0.1
remote:        Using mini_mime 1.0.2
remote:        Using mail 2.7.1
remote:        Using actionmailbox 6.0.1
remote:        Using actionmailer 6.0.1
remote:        Using actiontext 6.0.1
remote:        Using awesome_print 1.8.0
remote:        Using bcrypt 3.1.13
remote:        Using msgpack 1.3.1
remote:        Using bootsnap 1.4.5
remote:        Using bundler 1.17.3
remote:        Using execjs 2.7.0
remote:        Using multipart-post 2.1.1
remote:        Using faraday 0.17.0
remote:        Using ffi 1.11.1
remote:        Using hashie 3.6.0
remote:        Using jbuilder 2.9.1
remote:        Using jwt 2.2.1
remote:        Using method_source 0.9.2
remote:        Using multi_json 1.14.1
remote:        Using multi_xml 0.6.0
remote:        Using oauth2 1.4.2
remote:        Using omniauth 1.9.0
remote:        Using omniauth-oauth2 1.6.0
remote:        Using omniauth-auth0 2.2.0
remote:        Using omniauth-rails_csrf_protection 0.1.2
remote:        Using pg 1.1.4
remote:        Using puma 3.12.1
remote:        Using pundit 2.1.0
remote:        Using rack-proxy 0.6.5
remote:        Using thor 0.20.3
remote:        Using railties 6.0.1
remote:        Using sprockets 3.7.2
remote:        Using sprockets-rails 3.2.1
remote:        Using rails 6.0.1
remote:        Using rb-fsevent 0.10.3
remote:        Using rb-inotify 0.10.0
remote:        Using rename 1.0.6
remote:        Using sass-listen 4.0.0
remote:        Using sass 3.7.4
remote:        Using tilt 2.0.10
remote:        Using sass-rails 5.1.0
remote:        Using twilio-ruby 5.29.0
remote:        Using uglifier 4.2.0
remote:        Using webdack-uuid_migration 1.2.0
remote:        Using webpacker 4.0.7
remote:        Using webpacker-react 0.3.2
remote:        Bundle complete! 30 Gemfile dependencies, 77 gems now installed.
remote:        Gems in the groups development and test were not installed.
remote:        Bundled gems are installed into `./vendor/bundle`
remote:        Bundle completed (0.48s)
remote:        Cleaning up the bundler cache.
remote:        The dependency tzinfo-data (>= 0) will be unused by any of the platforms Bundler is installing for. Bundler is installing for ruby but the dependency is only for x86-mingw32, x86-mswin32, x64-mingw32, java. To add those platforms to the bundle, run `bundle lock --add-platform x86-mingw32 x86-mswin32 x64-mingw32 java`.
remote: -----> Installing node-v10.15.3-linux-x64
remote: -----> Installing yarn-v1.16.0
remote: -----> Detecting rake tasks
remote: -----> Preparing app for Rails asset pipeline
remote:        Running: rake assets:precompile
remote:        yarn install v1.16.0
remote:        [1/4] Resolving packages...
remote:        [2/4] Fetching packages...
remote:        info [email protected]: The platform "linux" is incompatible with this module.
remote:        info "[email protected]" is an optional dependency and failed compatibility check. Excluding it from installation.
remote:        [3/4] Linking dependencies...
remote:        warning " > @babel/[email protected]" has unmet peer dependency "@babel/core@^7.0.0-0".
remote:        warning "@babel/preset-react > @babel/[email protected]" has unmet peer dependency "@babel/core@^7.0.0-0".
remote:        warning "@babel/preset-react > @babel/[email protected]" has unmet peer dependency "@babel/core@^7.0.0-0".
remote:        warning "@babel/preset-react > @babel/[email protected]" has unmet peer dependency "@babel/core@^7.0.0-0".
remote:        warning "@babel/preset-react > @babel/[email protected]" has unmet peer dependency "@babel/core@^7.0.0-0".
remote:        warning "@babel/preset-react > @babel/plugin-transform-react-jsx > @babel/[email protected]" has unmet peer dependency "@babel/core@^7.0.0-0".
remote:        warning " > [email protected]" has unmet peer dependency "webpack@^4.0.0".
remote:        warning "webpack-dev-server > [email protected]" has unmet peer dependency "webpack@^4.0.0".
remote:        [4/4] Building fresh packages...
remote:        Done in 34.30s.
remote:        Compiling…
remote:        Compilation failed:
remote:
remote:
remote:  !
remote:  !     Precompiling assets failed.
remote:  !
remote:  !     Push rejected, failed to compile Ruby app.
remote:
remote:  !     Push failed
remote: Verifying deploy...

[tmoney100]

I'm getting the same thing on heroku:

remote:        Done in 34.30s.
remote:        Compiling…
remote:        Compilation failed:
remote:
remote:
remote:  !
remote:  !     Precompiling assets failed.
remote:  !
remote:  !     Push rejected, failed to compile Ruby app.

been trying different things for a week now. :/

My engines:

  "engines": {
    "node": "12.x.x",
    "yarn": "1.x.x"
  },

Weird thing is i can get assets to precompile locally, once but then if I clobber and try again it says i'm missing packages like @babel/core when I literally have done nothing besides assets:clobber. I'm running webpacker from init, nothing changed in the webpacker.yml or

# webpacker.yml
default: &default
  ...(default)...
  check_yarn_integrity: false
  webpack_compile_output: true

production:
  <<: *default

  # Production depends on precompilation of packs prior to booting for performance.
  compile: false

  # Extract and emit a css file
  extract_css: true

  # Cache manifest.json for performance
  cache_manifest: true

[szTheory]

This error can occur if you update the webpacker gem without updating the corresponding NPM packages. Make sure to run all the commands from https://github.com/rails/webpacker#upgrading

bundle update webpacker
rails webpacker:binstubs
yarn upgrade @rails/webpacker --latest
yarn upgrade webpack-dev-server --latest

[matthewgoodwin]

I have run all the updates, but I get the same error in webpacker 3.5:

remote:        yarn install v1.16.0
remote:        warning package.json: No license field
remote:        warning No license field
remote:        [1/4] Resolving packages...
remote:        [2/4] Fetching packages...
remote:        info [email protected]: The platform "linux" is incompatible with this module.
remote:        info "[email protected]" is an optional dependency and failed compatibility check. Excluding it from installation.
remote:        [3/4] Linking dependencies...
remote:        warning "@rails/webpacker > [email protected]" has unmet peer dependency "caniuse-lite@^1.0.30000697".
remote:        warning " > [email protected]" has unmet peer dependency "webpack@^2.2.0 || ^3.0.0".
remote:        warning "webpack-dev-server > [email protected]" has unmet peer dependency "webpack@^1.0.0 || ^2.0.0 || ^3.0.0".
remote:        [4/4] Building fresh packages...
remote:        Done in 26.30s.
remote:        Webpacker is installed 🎉 🍰
remote:        Using /tmp/build_785c758e3345988890566a863f7cfad0/config/webpacker.yml file for setting up webpack paths
remote:        Compiling…
remote:        Compilation failed:

[...]

[13] ./node_modules/css-loader??ref--2-2!./node_modules/postcss-loader/lib??ref--2-3!./node_modules/sass-loader/lib/loader.js??ref--2-4!./app/javascript/packs/stylesheets/application.scss 418 bytes [built]
remote:            + 9 hidden modules
remote:        
remote:        ERROR in ./node_modules/mapbox-gl/dist/mapbox-gl.css
remote:        Module build failed: ModuleBuildError: Module build failed: TypeError: Cannot read property 'toFixed' of undefined

[...]

remote:        Child extract-text-webpack-plugin node_modules/extract-text-webpack-plugin/dist node_modules/css-loader/index.js??ref--2-2!node_modules/postcss-loader/lib/index.js??ref--2-3!node_modules/sass-loader/lib/loader.js??ref--2-4!app/javascript/packs/stylesheets/application.scss:
remote:               [0] ./node_modules/css-loader??ref--2-2!./node_modules/postcss-loader/lib??ref--2-3!./node_modules/sass-loader/lib/loader.js??ref--2-4!./app/javascript/packs/stylesheets/application.scss 418 bytes {0} [built]
remote:                + 1 hidden module
remote:        
remote: 
remote:  !
remote:  !     Precompiling assets failed.
remote:  !
remote:  !     Push rejected, failed to compile Ruby app.
remote: 
remote:  !     Push failed

i've updated dependencies; i've looked though several posts; this is the closest post I find that mirrors my issue.. whats wrong here? I can post the full error if needed..
My app in development works fine; with no issues, but pushing to heroku; does not work..

//app/javascript/plugins/init_mapbox.js

import mapboxgl from 'mapbox-gl';
[..]
export { initMapbox };

// app/javascript/packs/application.js

import 'mapbox-gl/dist/mapbox-gl.css';

//package.json`

{
  "dependencies": {
    "@rails/webpacker": "3.5",
    "bootstrap": "^4.4.1",
    "jquery": "^3.4.1",
    "mapbox-gl": "^1.6.1",
    "popper.js": "^1.16.0"
  },
  "devDependencies": {
    "webpack-dev-server": "2.11.2"
  }
}

//application.html.erb

<%= stylesheet_link_tag 'application', media: 'all', 'data-turbolinks-track': 'reload' %>
<%= stylesheet_pack_tag 'application', 'data-turbolinks-track': 'reload' %>
<%= javascript_include_tag 'application' %> <!-- from app/assets/javascripts/application.js -->
<%= javascript_pack_tag 'application' %> <!-- from app/javascript/packs/application.js -->

please help!

[jakeNiemiec]

To everyone in this thread:

If your issue is not related to /bin/sh: 1: npm: not found, please create another issue with the details of your problem. In order to help debug, consider posting as much as you can of:

• ./package.json
• ./babel.config.js
• ./config/webpack/environment.js
• ./config/webpacker.yml
• ./app/javascript/packs/application.js (or the file related to the error)
• the full error message (even the parts you wouldn’t think of as relevant)

I am going to hide the errors that don't seem related to this issue. You may also find some heroku-specific solutions in: https://github.com/rails/webpacker/issues/395

[jakeNiemiec]

info [email protected]: The platform "linux" is incompatible with this module.
warning "@rails/webpacker > [email protected]" has unmet peer dependency "caniuse-lite@^1.0.30000697".

@matthewgoodwin You have several warnings, but this 👆 is probably the source of your problems. If you open a new issue, I can advise further.

[gathuku]

Am having this issue deploying rails 6 on heroku , Any solution?

Have tried defining engines in package.json but doest work

[ahowardm]

I was having the same issue with rails 6.0.2 and webpack and solved it adding two buildpacks to the heroku app (Settings > Buildpacks > Add buildpack):

1. heroku/nodejs
2. heroku/ruby

It is very important the order, at first I had ruby and then nodejs and it didn't work.

[gathuku]

@ahowardm Following your guide i was able to get it. But also had to disable rails default yarn install , since it was installing node packages and gems twice. see code

Rake::Task['yarn:install'].clear
namespace :yarn do
  desc "Disabling internal yarn install from Rails"
  task :install => [:environment] do
    puts "Disabling internal yarn install from Rails"
  end
end

Rake::Task['webpacker:yarn_install'].clear
namespace :webpacker do
  desc "Disabling internal yarn install from Rails"
  task :yarn_install => [:environment] do
    puts "Disabling internal yarn install from Rails"
  end
end

More https://dev.to/vvo/a-rails-6-setup-guide-for-2019-and-2020-hf5