Rocket.chat: Upload is not working

Created on 23 Sep 2016 · 32Comments · Source: RocketChat/Rocket.Chat

We are on latest snap package and 🚀 chat is at Version: 0.40.1

Here's the background:
It was working previously in 0.38 version. It stopped working after the upgrade of a snap package. Actually, I was fiddling with settings on AVATAR upload path and I remember setting to FILESYSTEM with /opt/avatar and I broke it. Since server itself was not starting up, I had to change things from backend through MongoDB console in rocketchat.settings collection ( reverted to GridFS).

We started experiencing this as well. Symptoms are almost the same but the scenarios are different.

When "upload" settings are tied to "Filesystem", the progress bar would freeze at 100% and nothing happens at all no matter the time. We get following in the log

ufs: cannot write file "zkb9tX9CKP8zLkX2W" (EROFS: read-only file system, open '/zkb9tX9CKP8zLkX2W.gif') { [Error: EROFS: read-only file system, open '/zkb9tX9CKP8zLkX2W.gif']
  errno: -30,
  code: 'EROFS',
  syscall: 'open',
  path: '/zkb9tX9CKP8zLkX2W.gif' }
Exception while invoking method 'ufsComplete' Error: EROFS: read-only file system, open '/zkb9tX9CKP8zLkX2W.gif'
    at Object.Future.wait (/snap/rocketchat-server/45/node_modules/fibers/future.js:449:15)
    at [object Object].ufsComplete (meteor://💻app/packages/jalik:ufs/ufs-methods.js:62:20)
    at [object Object].methodsMap.(anonymous function) (meteor://💻app/packages/rocketchat:lib/server/lib/debug.js:17:26)
    at maybeAuditArgumentChecks (meteor://💻app/packages/ddp-server/livedata_server.js:1711:12)
    at meteor://💻app/packages/ddp-server/livedata_server.js:711:19
    at [object Object]._.extend.withValue (meteor://💻app/packages/meteor/dynamics_nodejs.js:56:1)
    at meteor://💻app/packages/ddp-server/livedata_server.js:709:40
    at [object Object]._.extend.withValue (meteor://💻app/packages/meteor/dynamics_nodejs.js:56:1)
    at meteor://💻app/packages/ddp-server/livedata_server.js:707:46
    at Session.method (meteor://💻app/packages/ddp-server/livedata_server.js:681:23)
    at meteor://💻app/packages/ddp-server/livedata_server.js:551:43
    - - - - -
    at Error (native)
ufs: cannot delete temp file "/tmp/ufs/zkb9tX9CKP8zLkX2W" (ENOENT: no such file or directory, unlink '/tmp/ufs/zkb9tX9CKP8zLkX2W')

As for GridFS case, upload shows it worked fine but we get the blank placeholder of zero-byte / an empty file. However, nothing gets vomited in logs in this case.
screen shot 2016-09-23 at 11 17 29

file sharing bug

Source

kaji-bikash

👍1

Most helpful comment

yep, it works when setting the path to /var/snap/rocketchat-server/common/uploads, you just need to mkdir /var/snap/rocketchat-server/common/uploads first

maxlath on 13 Apr 2017

❤2 👍2 🎉1

All 32 comments

@geekgonecrazy Was this resolved in the #ubuntu-snap channel or is this still ongoing?

graywolf336 on 24 Sep 2016

Ongoing. Couple other reports in non snap installs. Trying to get to the bottom of it still

geekgonecrazy on 24 Sep 2016

I am experimenting the same problem using the latest #ubuntu-snap

rodripf on 26 Sep 2016

@rodripf experimenting or experiencing ? For former, thank you :) and for the latter, kudos for validating my case 👍

kaji-bikash on 26 Sep 2016

I am on snap 0.41 and I am experiencing this too.

rdd2 on 6 Oct 2016

I’m experiencing this issue too. Here are some details about the behaviour on my side:

I’m also running the latest snap. Avatar images and file uploads are both set to GridFS. In both cases, I get an empty file after upload (just like OP) and no warning or error message.

However, both file upload and avatar images work if I start the server directly in Node JS (without running it through Snap), e.g. like this:

cd /snap/rocketchat-server/current/
export ROOT_URL=http://localhost PORT=3000 MONGO_URL=mongodb://localhost:27017/parties
./bin/node main.js

So the issue seems very snap specific.

I noted that for me, uploading files only doesn’t work for _image files_ (e.g. pdf is okay). Furthermore, Avatar images work if I disable resizing them.

Does Rocket.Chat try to resize uploaded images? (Or run them through graphicsmagick for any other reason?) If yes, can this be disabled? Because that could be a workaround.

If the issue is really because of graphicsmagick, my wild guess would be that Rocket.Chat cannot find the gm executable in its Snap container.

To summarize, the situation on my server is like this:

| | Running through snap | Running directly through Node JS |
| --- | --- | --- |
| Avatar upload with resizing | No | Yes |
| Avatar upload without resizing | Yes | Yes |
| Image upload | No | Yes |
| Other File upload | Yes | Yes |

jGleitz on 8 Oct 2016

👍1

@jGleitz It looks like it uses $SNAP/usr/bin/convert.

vim stable/snapcraft.yaml

stage-packages:
  - imagemagick
organize:
   usr/bin/convert-im6: usr/bin/convert

falkowich on 8 Oct 2016

It looks like it uses $SNAP/usr/bin/convert.

Good hint. But if it’s using its own binary, then why is there a difference between running inside the snap and “outside”?

I thought Rocket.Chat would use gm because that’s mentioned in this bug:

Rocket.Chat uses GraphicsMagick for resizing, so you'd have to have that installed on your box.

Is that outdated info?

jGleitz on 8 Oct 2016

Both imagemagick and graphicsmagick provide the convert command. Rocket.Chat works with either one of them. We install imagemagick and include it in the snap.

Snaps are a isolated environment. Its packaged up with all of its dependencies. Also its restricted on which files and folders it can access. It can just access any file or folder it wishes. They are tightly controlled by the snap platform. This is one of the big benefits to using snaps.

So it appears for some reason its trying to use a file or folder it doesn't have access to on file upload.

geekgonecrazy on 9 Oct 2016

I can confirm @jGleitz's findings. Exact same symptoms for me.

rdd2 on 10 Oct 2016

In the latest snap it should be fixed.

We used to work with imagemagick or graphicsmagick. Something changed and if using imagemagick it seems to screw things up. So Graphicsmagick is required.

geekgonecrazy on 11 Oct 2016

And so it is. Thanks for your efforts.

rdd2 on 21 Oct 2016

I encountered that exact same issue.
Rocket.Chat Version: 0.54.2
Ubuntu 16.04 / snap install

Can't it be due to file system access restriction inherent to snaps?

maxlath on 12 Apr 2017

@maxlath if you have it setup to point to a filesystem path. Then yes absolutely. What path do you have specified?