Webpacker: Do not run yarn install on every assets:precompile

[Gargron]

At least add an option to disable that behaviour. Usually you have a separate step for making sure bundler/yarn dependencies are installed - I'd say in most cases. So this invocation is just wasting a few seconds in the best case, and in the worst case as I'm experiencing right now it's triggering post-install scripts every time.

[himdel]

IMHO dealing yarn should never even be webpacker's job. Don't tell me how to download my dependencies, just compile them when they are there. (Since most projects already had to deal with this and have their own solutions anyway.)

[gauravtiwari]

@Gargron @himdel That makes sense. I guess it should behave like bundler. This needs updating at two places - on Rails side and other on Webpacker - basically need to remove precompile hook enhancement.

[mipearson]

I tried working around this using the same workaround described at #440, but for some reason the yarn:install dep is added at some point after my own workaround is called.

Rake is confusing :(

[Gargron]

Afaik this is the line that does it: https://github.com/rails/webpacker/blob/master/lib/tasks/webpacker/compile.rake#L32 It also checks for something present in Rails 5.1, and I do wonder if Rails 5.1 also commits this mistake or not?

[gauravtiwari]

@Gargron Yep it's there too - 5.1+

[wagoid]

Since they check if the yarn:install task is defined before running it (here), I created this same task with an empty body, as a workaround.

namespace :yarn do
  task :install do
    # workaround to stop running yarn install after precompile
  end
end

[noelrappin]

@wagoid Hmm.. for me that runs both my workaround task and the existing task...

[gutierlf]

@wagoid, I also tried your workaround, only to find that it still runs the existing task. 🙁 I'm glad you offered an attempt though. 👍

So does anyone have another workaround? I'm stuck.

Thanks!

[wagoid]

Yeah, I thought that worked at first but it didn't :(
I also tried clearing the rake task, no success

[gutierlf]

@wagoid thanks for chiming in about your results too.

[dbalatero]

I'm having weird issues on Heroku with Rails 5.1, webpacker 3.0.1, and the nodejs + ruby buildpacks.

Whenever I deploy, the nodejs buildpack will run yarn install. Afterwards, the ruby buildpack will run rake assets:precompile, which will do a whole new yarn install, which doesn't seem to be picking up the cache from the prior run, and does a full package download again. If I heroku run bash after the fact and run yarn install manually, it downloads all the packages again as well.

I've been banging my head against the wall trying to get this to just download packages a single time and cache them between builds, but no luck. Anyone else have a similar issue or fix?

I wish I could just yarn install once during the nodejs buildpack and then webpacker would pickup that all the modules are there and everything is ok…

[gauravtiwari]

Is NODE_MODULES_CACHE set to true? See here: https://devcenter.heroku.com/articles/nodejs-support#cache-behavior

[shaicoleman]

Deleting bin/yarn in the app will disable yarn

[dbalatero]

Is NODE_MODULES_CACHE set to true? See here: https://devcenter.heroku.com/articles/nodejs-support#cache-behavior

@gauravtiwari yeah it's set to true!

Deleting bin/yarn in the app will disable yarn

@shaicoleman Interesting… I will go down this path and see what happens.

[gutierlf]

I also came to this issue when trying to deploy to Heroku. I just tried setting NODE_MODULES_CACHE to true and deleting bin/yarn, but neither worked for me.

[dbalatero]

@gutierlf I got it to work by deleting bin/yarn. Once I deleted that binary, webpacker's compile step stopped trying to install a second time.

I'm using the nodejs and ruby buildpacks in that order, with these vars:

remote:        NPM_CONFIG_LOGLEVEL=error
remote:        NPM_CONFIG_PRODUCTION=true
remote:        NODE_VERBOSE=false
remote:        NODE_ENV=production
remote:        NODE_MODULES_CACHE=true

When my nodejs buildpack hit, it took care of doing yarn install (and also cached the results for future deploys, which is :+1:).

As well, I'm setting explicit versions in my package.json (which I don't think matters, but I'm mentioning for completeness):

{
  "engines": {
    "node": "8.5.0",
    "npm": "5.3.0",
    "yarn": "^1.0.2"
  }
}

I had to move some babel plugins from devDependencies to my dependencies section to get the actual compilation to work. There's a discussion in #117 that says any build tools need to be put into dependencies so they get installed on Heroku. This is likely because NPM_CONFIG_PRODUCTION is set to true, which must run yarn install --production – which would not pull in devDependencies. Without those, rake assets:precompile will barf.

Let me know if this info helps.

[gutierlf]

@dbalatero thanks for sharing so many details about what worked for you. I'll try later on and hopefully have the same success you did. In any case, I'll report back.

[gutierlf]

Ok, my Heroku deploy is working now 🎉, but I'm not sure which was the change that really fixed things 🤔. Following @dbalatero's advice, I set the five vars and used the nodejs and ruby buildpacks. I had previously tried using those buildpacks without success, so my attempts just before @dbalatero's comment had been done without specifying the buildpacks (which I assumed meant that Heroku would choose a smart default). I did not remove bin/yarn. And tra-la-la, success!

My current guess is that it has to do with the NODE_MODULES_CACHE=true setting, but I didn't try narrowing things down to see if that guess holds water.

Anyhow, thanks @dbalatero.

[dbalatero]

@gutierlf glad to hear it's working!

What I'd love to be able to do is _only_ use the ruby buildpack on Heroku. I think the only reason that I'm using the nodejs buildpack is so that I can get higher versions of Node and Yarn, as webpacker I believe installs node 6.x and yarn 0.22.x, and I'm using 8.5.0 and 1.1.0 respectively on my dev machine. If there was a way to hint to webpacker which versions we actually want to install when doing a production assets:precompile, I believe that would completely eliminate the dependency on the nodejs buildpack, and Rails to Heroku deploys would just be seamless again.

This might be getting off topic though at this point…

[wickkidd]

FWIW, I was able to bypass webpacker's compile-related tasks (and underlying hard dependency on yarn) by simply making my own task derived from webpacker's and setting the environment variable WEBPACKER_PRECOMPILE to false.

Rake::Task["assets:precompile"].enhance do
  $stdout.sync = true

  def ensure_log_goes_to_stdout
    old_logger = Webpacker.logger
    Webpacker.logger = ActiveSupport::Logger.new(STDOUT)
    yield
  ensure
    Webpacker.logger = old_logger
  end

  namespace :webpacker do
    desc "Compile JavaScript packs using webpack for production with digests"
    task custom_compile: :environment do
      ensure_log_goes_to_stdout do
        if Webpacker.compile
          # Successful compilation!
        else
          # Failed compilation
          exit!
        end
      end
    end
  end

  Rake::Task["webpacker:custom_compile"].invoke
end

Granted, this (as well as some other approaches I've come across) aren't what I would call ideal, but I believe that the risk is fairly low:

1. If "Webpacker.compile" changes, I'll have to adjust.
2. There's no verification of node or the binstubs, however, since I'm handling node and npm installation separately I'm not really concerned with them.
3. Things I haven't thought of ;)

[wbotelhos]

Thank you @wagoid and @shaicoleman , it worked! <3

[atz]

Since they check if the yarn:install task is defined before running it (here), I created this same task with an empty body, as a workaround.

namespace :yarn do
  task :install do
    # workaround to stop running yarn install after precompile
  end
end

The way to nullify an existing rake task prior to redefinition (which otherwise appends, as commenters found out) is:

Rake::Task["yarn:install"].clear

[swistak]

I'm having this problem in Rails 6 pre. Why is yarn:install called at all when precompilng assets?
Why are assets concerned with dependency management?

PS. Working workaround (to be added at the end of a Rakefile):

Rake::Task["yarn:install"].clear
namespace :yarn do
  task :install do
    # Someone decided to install yarn every time asset:precompile is called for some strange reason.
    # It's less then optimal - as it prevents us from doing it in a separate step during deploy with specific options.
    # So we clear the old task and override it with an empty one.
  end
end

[wbotelhos]

@swistak Did you tried to just remove bin/yarn?

[swistak]

But I need a yarn to in a project and I'm installing it in a different step with custom flags.

[nitsujri]

For anyone coming across this today:

rm bin/yarn

# yarn_install.rake
Rake::Task['yarn:install'].clear
namespace :yarn do
  task :install do
    # Redefine as empty
  end
end

This works for our deployment scheme by rm'ing the bin/yarn in a specific step.

[heaven]

Was this ever fixed? The workarounds don't work, the precompile task even fails to start because of Yarn requires Node.js 4.0 or higher to be installed. While 12.6.3 is actually installed via NVM.

[andreorvalho]

setting NODE_ENV='development' fixed the problem when running yarn install and getting all necessary dev dependencies. Not sure this is the solution for your problem. But it helped me solve a similar one and hope it helps somebody else