Webpacker: Deploy on AWS Beanstalk

[sealocal]

I'm breaking out a discussion (from #116) for the best approach deployment of Rails with webpacker and yarn on AWS Elastic Beanstalk (and maybe EC2).

I'm using the latest Ruby platform on AWS EBS:

64bit Amazon Linux 2016.09 v2.3.1 running Ruby 2.3 (Puma)

I encounter errors upon deploy with:

$ eb deploy

Some key log messages are that yarn is not detected:

Yarn executable was not detected in the system.
Download Yarn at https://yarnpkg.com/en/docs/install

and a missing webpack node module:

sh: ./node_modules/webpack/bin/webpack.js: No such file or directory

Using Rails 5.0.1 and webpacker 1.0, the rake assets:precompile fails due the missing webpack module. However, I know that the reason the module is missing is because none of my modules are installed, because yarn is not available.

I solved this by installing yarn globally on the server, but I would prefer that I could check for a missing yarn and install prior to rake assets:precompile.

Is anyone else trying to solve the same problem? Is it overkill to ask that yarn be installed upon deploy?

[p0wl]

In my opinion yarn is a prerequisite for using webpacker (like nodejs), so you should install it in your app setup (e.g. dockerfile).

Otherwise, webpacker would have to detect how to install yarn on your system (which package manager to use, ...).

[sealocal]

I do not expect webpacker to implement installation of yarn, but just hoping that others who are looking to solve deployment with the same parameters will chime in here. I'll close the issue.

I'm not familiar with Docker, but I will take a look. Thanks!

[p0wl]

You can customize your elastic beanstalk app via .ebextensions, maybe http://stackoverflow.com/questions/41657226/customize-aws-elasticbeanstalk-nodejs-install-use-yarn helps =)

[sealocal]

Thank you, that is correct, but I have used and tweaked that script many times with no success.

[sealocal]

@gauravtiwari @p0wl As it turns out, my problem was installing node as a dependency of yarn!

I used the .ebextensions files, but tweaked the yarn installation bash commands to install node with sudo, and it works!

Just leaving this here in case it helps someone else with AWS deployment.

[gauravtiwari]

Great 👍

[josephecombs]

In case anyone else sees this, there is a good .ebextension listed here for resolving this issue:

https://github.com/shakacode/react_on_rails/blob/3269da94455813a4ebff49c6a95a623fdcc61713/docs/additional-reading/elastic-beanstalk.md

[hugodias]

@sealocal can you share your .ebextensions file?

[sealocal]

@josephecombs Thanks for circling back to post that. I looked many times for something that used the command key - I will test out that approach on my setup when I have the opportunity.

[josephecombs]

I actually made a mistake with that link - this is the full ebextensions file: https://github.com/shakacode/react_on_rails/blob/master/docs/additional-reading/elastic-beanstalk.md

[sealocal]

@hugodias This is the eb config I created.

.ebextensions/01_yarn.config:

files:
  # If this file is edited, it must be removed from EC2 instance prior to deploy.
  "/opt/elasticbeanstalk/hooks/appdeploy/pre/09_yarn_install.sh" :
    mode: "000775"
    owner: root
    group: users
    content: |
      #!/usr/bin/env bash

      set -xe

      EB_APP_STAGING_DIR=$(/opt/elasticbeanstalk/bin/get-config container -k app_staging_dir)
      EB_APP_USER=$(/opt/elasticbeanstalk/bin/get-config container -k app_user)

      echo "I am: `whoami`"
      echo "App user is $EB_APP_USER"

      # If yarn is not detected, install it.
      if which yarn; then
        echo "Skipping installation of yarn -- yarn already installed."
        echo "yarn --version: `yarn --version`"
      else
        echo "which yarn: `which yarn`"
        echo "Yarn is not installed and accessible."
        echo "Installing yarn..."
        # Consider that the EC2 instance is managed by AWS Elastic Beanstalk.
        # Changes made via SSH WILL BE LOST if the instance is replaced by auto-scaling.
        # QUESTION: Will this script be run on new instances that are created by auto-scaling?
        # QUESTION: Should installation be moved to a rake task?

        # Download the yarn repo
        sudo wget https://dl.yarnpkg.com/rpm/yarn.repo -O /etc/yum.repos.d/yarn.repo
        # Confirm that it downloaded
        file /etc/yum.repos.d/yarn.repo

        # If node is not detected, install it.
        if [ `node --version` == 'v6.10.0' ]; then
          echo "Skipping installation of node -- node already installed."
          echo "node --version: `node --version`"
        else
          echo "Installing Node v6.10.0 ..."
          # Download the Node v6 setup script
          curl --location https://rpm.nodesource.com/setup_6.x > /home/ec2-user/node_install.sh
          # Confirm that it downloaded
          file /home/ec2-user/node_install.sh
          # Run the Node v6 setup script
          sudo bash /home/ec2-user/node_install.sh
          # Install nodejs
          sudo yum install -y nodejs
          node --version
          echo "... and finished installing Node v6.10.0"
        fi

        # install yarn
        sudo yum install -y yarn
        yarn --version

        echo "... and finished installing yarn."
      fi

      echo "Change directory to $EB_APP_STAGING_DIR"
      cd $EB_APP_STAGING_DIR

      # yarn install
      echo "Running yarn install."
      ./bin/yarn install

[MatthiasRMS]

@sealocal @josephecombs I've tried both of your yarn config file, and it seems to work fine to install it. However, I still get a 404 when the app tries to fetch the packs. It looks like webpack-dev-server didn't start, so it doesn't serves the JS files to the app.
How did you start webpack-dev-server on elastic beanstalk ?

Thanks

[sealocal]

@MatthiasRMS webpack-dev-server is intended for use in development - it watches for changes in the javascript that you're building with webpack - e.g., if you change a line of code in a component. (see https://github.com/rails/webpacker#development).

If your'e seeing a 404 for a request to a pack, then your pack doesn't exist. It hasn't been successfully compiled via the webpack binstub. Check the activity logs just after you deploy.

[tetz2442]

I'm also receiving 404s on webpack files when deploying to EB. From looking at the logs, it looks like webpacker is compiling just fine. Has anyone else run into this issue?

[josephecombs]

Are you on rails 5.1 @tetz2442? Are other assets being served fine? If so this seems like maybe a path reference issue, can you try referring to the packs in erb as, for example <%= stylesheet_pack_tag 'vendor_pack' %>

[sealocal]

It could be worth verifying that the pack is in the public directory, and double-check your choice to serve static assets from Rails.

[tetz2442]

@josephecombs I'm on rails 5.1. Everything is working fine, except for the webpack files. I reference files with <%= javascript_pack_tag 'editor' %>.

What's strange is the url it generates 404s, but if I add in /public before /packs like so /public/packs/editor-64ec86898e5722f4651f.js it works fine.

And this is only on EB, everything locally works fine.

[gauravtiwari]

@tetz2442 Guess you would need to tell rails to serve public files:

# in production.rb
config.public_file_server.enabled = true

[joshleichtung]

@tetz2442 I had the same issue. I think the problem is not with webpacker or rails, but with nginx static file serving settings. I fixed it by adding a location directive to the nginx config specifically mapping the /public/packs.

Here is the config file I added to the .ebextensions folder in the root of my app. Depending on how your Elastic Beanstalk instance is set up, you might need to make changes. Comments are in the file regarding important lines.

[tetz2442]

Once I followed @gauravtiwari advice and told rails to serve public files, the pack files didn't 404 anymore

[MSchmidt]

Since this was specifically about Elastic Beanstalk, you should actually set the RAILS_SERVE_STATIC_FILES environment variable.

eb setenv RAILS_SERVE_STATIC_FILES=enabled

[ajsharp]

I'd recently upgraded to rails 5.1, and I was getting this error message the Command "webpack" not found error message while building assets on elastic beanstalk. The issue for me was I had fsevents in my devDependencies, and for some reason that was breaking the build on AWS. I removed that from my dependencies and the build succeeded. Unfortunately that error message seems to be a catch-all, because webpack was installed on the system.

[pyrabbit]

I am not sure how relevant this is but I had to change the last line of .ebextensions/01_yarn.config: from ./bin/yarn install to yarn install. I noticed I didn't have a yarn binary in my applications bin folder and after running which yarn on my EC2 instance it was pointing at /usr/local/bin/yarn

[januszm]

RAILS_SERVE_STATIC_FILES is absolutely not a solution in production environments, looks like your HTTP (nginx) server does not know about the location /packs. Default configuration includes the public assets location only.

Unfortunately, as usual with Elastic Beanstalk, you have to wait for platform updates, and before that happens, you can always use your own custom configuration using the .ebextensions

[damien-roche]

Truly one of the most horrendous platforms. The solution: don't deploy to Beanstalk. Lost 2 days so far and not even precompiled assets.

[ctrlaltdylan]

I've been using EB with Rails & Webpacker for some time now, but it appears that my packs size has gotten to the point where it throttles the CPU on every deploy.

I'm going to look into Docker usage instead of .ebextensions, as well as having the CI handle the asset build process. It's just too much for even a t2-medium sitting at 5% CPU usage before deployment.

[rkiller]

@ctrlaltdylan I'm seeing the same but my pack size is not even that large. Deploying on a t2.medium is timing out the deployment process but ultimately deploys correctly...

[ctrlaltdylan]

@rkiller since I wrote this comment I've switched to my CI handling the assets compilation & webpacking. That particular step still fails from time to time on a docker container, but at least it's not causing downtime on the EBS app.