Mastodon: Error thrown by browserify causes assets:precompile to abort

[webbj74]

I am setting up an instance for the first time. When I ran:

$ RAILS_ENV=production bundle exec rails assets:precompile

It aborted after a while with:

~~~~~bunch o assets~~~~
I, [2017-04-15T19:36:40.396423 #8236]  INFO -- : Writing /home/mastodon/live/public/assets/void-767af9b741bc6b218b558bea8a5f430d3275a09756188e1e2eb1845a04bdb828.png
Killed

Re-running, showed an exception:

$ RAILS_ENV=production bundle exec rails assets:precompile
rails aborted!
BrowserifyRails::BrowserifyError: Error while running `/home/mastodon/live/node_modules/.bin/browserify --transform [ babelify --presets [ es2015 react ] ] --extension=".jsx" --list -o "/home/mastodon/live/tmp/cache/browserify-rails/output20170415-8336-d1paw6" -`:

Killed

Attached the backtrace. This happened when on @28606d730 and again after updating to @fa08b5079d

---

• [x] I searched or browsed the repo’s other issues to ensure this is not a duplicate.

[Gargron]

Are you running out of memory? Do you have any swap?

[mcat-ee]

I'm getting this error on a fresh Vultr Ubuntu instance.

2 GB of RAM, no other services running, and no swap allocated.

[mcat-ee]

It'd be cool to have version tags to enable rolling back - I'm considering rolling back to get the instance online but right now i'd be doing that to arbitrary commits.

Edit: my bad

[wxcafe]

there are version tags : https://github.com/tootsuite/mastodon/releases and it looks like a OOM-kill yeah

[mcat-ee]

I'm not sure it's an OOM-kill; I bumped the available RAM up to 4 GB and added a 4GB swapfile and I'm still receiving the error. For comparison a few weeks ago I could run Mastodon on a similarly-specced desktop.

[mcat-ee]

I span up a new VPS instance running a Centos image - Mastodon works completely out of the box. @webbj74 what operating system are you deploying Mastodon on?

[wxcafe]

I can guarantee you mastodon works on ubuntu and debian. I have no idea where this might have come from. @webbj74 can we close this?

[CACTANAKA]

Just information.

Maybe I face same issue at Mastodon v1.2 on MS-Azure CentOS7.
Some case, Cloud providers OS image is not so good.
(In this case, maybe npm logged error messages.)
I could build v1.2 on Local machine's CentOS7(Install type : Infra).

Like above, maybe Ubuntu is good.

I do following command before Re-build/upgrade.

RAILS_ENV=production bundle exec rails assets:clean

[webbj74]

I'm getting this with an Ubuntu 16.04.2 x64 VM with 512 MB Memory & 20 GB Disk hosted in NYC.

This evening I updated to v1.2.2 and trying assets:precomile again failed. After the proc was oom-killed:

$ free
              total        used        free      shared  buff/cache   available
Mem:         500100       55284      362852       20128       81964      395516
Swap:             0           0           0

[mcat-ee]

@wxcafe I'm not saying it doesn't work on Ubuntu/Debian, i'm saying there is something here worth investigating and documenting for future deployments/troubleshooting.

[webbj74]

Here's a screen capture showing straces of the execution along with top & free in separate panes
https://youtu.be/QPNSmTqVkUo

I can post the strace logs if you think they would be useful. My background isn't in rails, so let me know if there's a better way to get to the root of the problem.

[ddevault]

Definitely should not require more than 2G to compile static assets. Terrible choice of toolchain if that's necessary.

[wxcafe]

@SirCmpwn thanks for your contribution to fixing this issue.

[ddevault]

I'm proposing that the toolchain needs to be replaced, you prick. I can't exactly do that in 5 minutes and it merits discussion first.

[wxcafe]

This is definitely not the right place for this. Thank you for your contribution to my bio :)

[ddevault]

"Toolchain requires heaps of memory to run and this causes problems for people"

"Maybe we should replace the toolchain"

Yep, wrong place.

[Gargron]

@SirCmpwn This is completely inacceptable.

But Browserify is pretty terrible.

[ddevault]

Well, definitely not going to help now. Every time I come within spitting distance of Mastodon's GitHub I regret it.

[danhunsaker]

Do what you feel is right, but it is more useful to start a new issue for ... Well, a new issue. And "replace the toolchain" is a new issue compared to "having issues that might be memory-related?", even if the two are related issues.

[webbj74]

One thing I didn't mention explicitly (though my free output mentions) I don't have swap enabled. I did see in #672 that swap was recommended specifically for this case. Swapping to disk on a prod webserver seems like a really bad idea to me, but I will try temporarily flipping some on to get the assets generated.

[wxcafe]

swap is recommended if you don't have enough RAM. As this seems to not be the problem in your case, I don't think this will solve it.

Can you check you don't have a per-process RAM usage limit, or an opened-files limit, or some other thing of this kind?

[nolanlawson]

Can this be closed since we now use Webpack instead of Browserify?

[wxcafe]

Yeah probably