Rocket.chat: Snap fails to start on fresh install

[maxidorius]

Your Rocket.Chat version: 0.48.2 (latest snap release at time of writing)

Wanting to give Rocket.chat a spin, I did a fresh install but the server is simply not starting.
There are few recent issues that suggest to do an apt-get dist-upgrade. but everything is already installed here.
I followed the snap instruction on the Download page: snap install rocketchat-server

Logs of the server:

max@max-laptop-linux ~ $ sudo journalctl -u snap.rocketchat-server.rocketchat-server
-- Logs begin at Fri 2017-01-06 18:44:42 CET, end at Fri 2017-01-06 18:49:28 CET. --
Jan 06 18:44:44 max-laptop-linux systemd[1]: Started Service for snap application rocketchat-server.rocketchat-server.
Jan 06 18:44:44 max-laptop-linux snap[1540]: cannot change profile for the next exec call: No such file or directory
Jan 06 18:44:44 max-laptop-linux systemd[1]: snap.rocketchat-server.rocketchat-server.service: Main process exited, code=exited, status=1/FAILURE
Jan 06 18:44:44 max-laptop-linux systemd[1]: snap.rocketchat-server.rocketchat-server.service: Unit entered failed state.
Jan 06 18:44:44 max-laptop-linux systemd[1]: snap.rocketchat-server.rocketchat-server.service: Failed with result 'exit-code'.
Jan 06 18:44:44 max-laptop-linux systemd[1]: snap.rocketchat-server.rocketchat-server.service: Service hold-off time over, scheduling restart.
Jan 06 18:44:44 max-laptop-linux systemd[1]: Stopped Service for snap application rocketchat-server.rocketchat-server.
Jan 06 18:44:44 max-laptop-linux systemd[1]: Started Service for snap application rocketchat-server.rocketchat-server.
Jan 06 18:44:44 max-laptop-linux snap[1803]: cannot change profile for the next exec call: No such file or directory
Jan 06 18:44:44 max-laptop-linux systemd[1]: snap.rocketchat-server.rocketchat-server.service: Main process exited, code=exited, status=1/FAILURE
Jan 06 18:44:44 max-laptop-linux systemd[1]: snap.rocketchat-server.rocketchat-server.service: Unit entered failed state.
Jan 06 18:44:44 max-laptop-linux systemd[1]: snap.rocketchat-server.rocketchat-server.service: Failed with result 'exit-code'.
Jan 06 18:44:45 max-laptop-linux systemd[1]: snap.rocketchat-server.rocketchat-server.service: Service hold-off time over, scheduling restart.
Jan 06 18:44:45 max-laptop-linux systemd[1]: Stopped Service for snap application rocketchat-server.rocketchat-server.
Jan 06 18:44:45 max-laptop-linux systemd[1]: Started Service for snap application rocketchat-server.rocketchat-server.
Jan 06 18:44:45 max-laptop-linux snap[1972]: cannot change profile for the next exec call: No such file or directory
Jan 06 18:44:45 max-laptop-linux systemd[1]: snap.rocketchat-server.rocketchat-server.service: Main process exited, code=exited, status=1/FAILURE
Jan 06 18:44:45 max-laptop-linux systemd[1]: snap.rocketchat-server.rocketchat-server.service: Unit entered failed state.
Jan 06 18:44:45 max-laptop-linux systemd[1]: snap.rocketchat-server.rocketchat-server.service: Failed with result 'exit-code'.
Jan 06 18:44:45 max-laptop-linux systemd[1]: snap.rocketchat-server.rocketchat-server.service: Service hold-off time over, scheduling restart.
Jan 06 18:44:45 max-laptop-linux systemd[1]: Stopped Service for snap application rocketchat-server.rocketchat-server.
Jan 06 18:44:45 max-laptop-linux systemd[1]: Started Service for snap application rocketchat-server.rocketchat-server.
Jan 06 18:44:45 max-laptop-linux snap[2060]: cannot change profile for the next exec call: No such file or directory
Jan 06 18:44:45 max-laptop-linux systemd[1]: snap.rocketchat-server.rocketchat-server.service: Main process exited, code=exited, status=1/FAILURE
Jan 06 18:44:45 max-laptop-linux systemd[1]: snap.rocketchat-server.rocketchat-server.service: Unit entered failed state.
Jan 06 18:44:45 max-laptop-linux systemd[1]: snap.rocketchat-server.rocketchat-server.service: Failed with result 'exit-code'.
Jan 06 18:44:45 max-laptop-linux systemd[1]: snap.rocketchat-server.rocketchat-server.service: Service hold-off time over, scheduling restart.
Jan 06 18:44:45 max-laptop-linux systemd[1]: Stopped Service for snap application rocketchat-server.rocketchat-server.
Jan 06 18:44:45 max-laptop-linux systemd[1]: Started Service for snap application rocketchat-server.rocketchat-server.
Jan 06 18:44:45 max-laptop-linux snap[2122]: cannot change profile for the next exec call: No such file or directory
Jan 06 18:44:45 max-laptop-linux systemd[1]: snap.rocketchat-server.rocketchat-server.service: Main process exited, code=exited, status=1/FAILURE
Jan 06 18:44:45 max-laptop-linux systemd[1]: snap.rocketchat-server.rocketchat-server.service: Unit entered failed state.
Jan 06 18:44:45 max-laptop-linux systemd[1]: snap.rocketchat-server.rocketchat-server.service: Failed with result 'exit-code'.
Jan 06 18:44:45 max-laptop-linux systemd[1]: snap.rocketchat-server.rocketchat-server.service: Service hold-off time over, scheduling restart.
Jan 06 18:44:45 max-laptop-linux systemd[1]: Stopped Service for snap application rocketchat-server.rocketchat-server.
Jan 06 18:44:45 max-laptop-linux systemd[1]: snap.rocketchat-server.rocketchat-server.service: Start request repeated too quickly.
Jan 06 18:44:45 max-laptop-linux systemd[1]: Failed to start Service for snap application rocketchat-server.rocketchat-server.
max@max-laptop-linux ~ $ 

Logs of the DB server:

max@max-laptop-linux ~ $ sudo journalctl -u snap.rocketchat-server.rocketchat-mongo
-- Logs begin at Fri 2017-01-06 18:44:42 CET, end at Fri 2017-01-06 18:53:22 CET. --
Jan 06 18:44:44 max-laptop-linux systemd[1]: Started Service for snap application rocketchat-server.rocketchat-mongo.
Jan 06 18:44:44 max-laptop-linux snap[1543]: cannot change profile for the next exec call: No such file or directory
Jan 06 18:44:44 max-laptop-linux systemd[1]: snap.rocketchat-server.rocketchat-mongo.service: Main process exited, code=exited, status=1/FAILURE
Jan 06 18:44:44 max-laptop-linux systemd[1]: snap.rocketchat-server.rocketchat-mongo.service: Unit entered failed state.
Jan 06 18:44:44 max-laptop-linux systemd[1]: snap.rocketchat-server.rocketchat-mongo.service: Failed with result 'exit-code'.
Jan 06 18:44:44 max-laptop-linux systemd[1]: snap.rocketchat-server.rocketchat-mongo.service: Service hold-off time over, scheduling restart.
Jan 06 18:44:44 max-laptop-linux systemd[1]: Stopped Service for snap application rocketchat-server.rocketchat-mongo.
Jan 06 18:44:44 max-laptop-linux systemd[1]: Started Service for snap application rocketchat-server.rocketchat-mongo.
Jan 06 18:44:44 max-laptop-linux snap[1792]: cannot change profile for the next exec call: No such file or directory
Jan 06 18:44:44 max-laptop-linux systemd[1]: snap.rocketchat-server.rocketchat-mongo.service: Main process exited, code=exited, status=1/FAILURE
Jan 06 18:44:44 max-laptop-linux systemd[1]: snap.rocketchat-server.rocketchat-mongo.service: Unit entered failed state.
Jan 06 18:44:44 max-laptop-linux systemd[1]: snap.rocketchat-server.rocketchat-mongo.service: Failed with result 'exit-code'.
Jan 06 18:44:45 max-laptop-linux systemd[1]: snap.rocketchat-server.rocketchat-mongo.service: Service hold-off time over, scheduling restart.
Jan 06 18:44:45 max-laptop-linux systemd[1]: Stopped Service for snap application rocketchat-server.rocketchat-mongo.
Jan 06 18:44:45 max-laptop-linux systemd[1]: Started Service for snap application rocketchat-server.rocketchat-mongo.
Jan 06 18:44:45 max-laptop-linux snap[1976]: cannot change profile for the next exec call: No such file or directory
Jan 06 18:44:45 max-laptop-linux systemd[1]: snap.rocketchat-server.rocketchat-mongo.service: Main process exited, code=exited, status=1/FAILURE
Jan 06 18:44:45 max-laptop-linux systemd[1]: snap.rocketchat-server.rocketchat-mongo.service: Unit entered failed state.
Jan 06 18:44:45 max-laptop-linux systemd[1]: snap.rocketchat-server.rocketchat-mongo.service: Failed with result 'exit-code'.
Jan 06 18:44:45 max-laptop-linux systemd[1]: snap.rocketchat-server.rocketchat-mongo.service: Service hold-off time over, scheduling restart.
Jan 06 18:44:45 max-laptop-linux systemd[1]: Stopped Service for snap application rocketchat-server.rocketchat-mongo.
Jan 06 18:44:45 max-laptop-linux systemd[1]: Started Service for snap application rocketchat-server.rocketchat-mongo.
Jan 06 18:44:45 max-laptop-linux snap[2062]: cannot change profile for the next exec call: No such file or directory
Jan 06 18:44:45 max-laptop-linux systemd[1]: snap.rocketchat-server.rocketchat-mongo.service: Main process exited, code=exited, status=1/FAILURE
Jan 06 18:44:45 max-laptop-linux systemd[1]: snap.rocketchat-server.rocketchat-mongo.service: Unit entered failed state.
Jan 06 18:44:45 max-laptop-linux systemd[1]: snap.rocketchat-server.rocketchat-mongo.service: Failed with result 'exit-code'.
Jan 06 18:44:45 max-laptop-linux systemd[1]: snap.rocketchat-server.rocketchat-mongo.service: Service hold-off time over, scheduling restart.
Jan 06 18:44:45 max-laptop-linux systemd[1]: Stopped Service for snap application rocketchat-server.rocketchat-mongo.
Jan 06 18:44:45 max-laptop-linux systemd[1]: Started Service for snap application rocketchat-server.rocketchat-mongo.
Jan 06 18:44:45 max-laptop-linux snap[2124]: cannot change profile for the next exec call: No such file or directory
Jan 06 18:44:45 max-laptop-linux systemd[1]: snap.rocketchat-server.rocketchat-mongo.service: Main process exited, code=exited, status=1/FAILURE
Jan 06 18:44:45 max-laptop-linux systemd[1]: snap.rocketchat-server.rocketchat-mongo.service: Unit entered failed state.
Jan 06 18:44:45 max-laptop-linux systemd[1]: snap.rocketchat-server.rocketchat-mongo.service: Failed with result 'exit-code'.
Jan 06 18:44:45 max-laptop-linux systemd[1]: snap.rocketchat-server.rocketchat-mongo.service: Service hold-off time over, scheduling restart.
Jan 06 18:44:45 max-laptop-linux systemd[1]: Stopped Service for snap application rocketchat-server.rocketchat-mongo.
Jan 06 18:44:45 max-laptop-linux systemd[1]: snap.rocketchat-server.rocketchat-mongo.service: Start request repeated too quickly.
Jan 06 18:44:45 max-laptop-linux systemd[1]: Failed to start Service for snap application rocketchat-server.rocketchat-mongo.
max@max-laptop-linux ~ $ 

Apt output:

max@max-laptop-linux ~ $ sudo apt-get update
Ign:1 http://dl.google.com/linux/chrome/deb stable InRelease
Hit:2 http://dl.google.com/linux/chrome/deb stable Release                                                                                                                                                            
Get:3 http://ftp.kfki.hu/linux/ubuntu xenial InRelease [247 kB]                                                                                                                                                                              
Ign:4 http://mirrors.serverhost.ro/mint/packages serena InRelease                                                                                                                                                                            
Hit:6 http://mirrors.serverhost.ro/mint/packages serena Release                                                                                                                                                                              
Get:7 http://security.ubuntu.com/ubuntu xenial-security InRelease [102 kB]                                                                                                                                                            
Hit:8 http://ppa.launchpad.net/wine/wine-builds/ubuntu xenial InRelease                                                                                                                                                                      
Hit:9 http://archive.canonical.com/ubuntu xenial InRelease                                                                                                                                                                                   
Get:10 http://ftp.kfki.hu/linux/ubuntu xenial-updates InRelease [102 kB]                                                                                                                                                        
Get:12 http://ftp.kfki.hu/linux/ubuntu xenial-backports InRelease [102 kB]                                                                                                  
Hit:13 https://repo.morph027.de/nextcloud-client xenial InRelease                                                                                                
Hit:14 http://download.virtualbox.org/virtualbox/debian xenial InRelease 
Hit:15 http://repo.steampowered.com/steam precise InRelease              
Fetched 553 kB in 1s (478 kB/s)
Reading package lists... Done
max@max-laptop-linux ~ $ sudo apt-get upgrade
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Calculating upgrade... Done
0 to upgrade, 0 to newly install, 0 to remove and 0 not to upgrade.
max@max-laptop-linux ~ $ sudo apt-get dist-upgrade
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Calculating upgrade... Done
0 to upgrade, 0 to newly install, 0 to remove and 0 not to upgrade.
max@max-laptop-linux ~ $ 

[Satotripe]

You have dist-upgrade your distribution ?

[maxidorius]

Yes, I also have provided the output in the OP.

[geekgonecrazy]

How much available memory do you have?

[maxidorius]

4GB

max@max-laptop-linux ~ $ free -m
              total        used        free      shared  buff/cache   available
Mem:           7915        2686        3348         629        1880        4303
Swap:          8122           0        8122
max@max-laptop-linux ~ $

[geekgonecrazy]

I did just notice this appears to be Linux Mint? Being based on Ubuntu... You wouldn't think there would be issues. Can you give the version of snap using: snap --version or snap version ?

[maxidorius]

I had the same thoughts... For the sake of clarity, here is the version of Mint I am using:

max-laptop-linux ~ # cat /etc/lsb-release 
DISTRIB_ID=LinuxMint
DISTRIB_RELEASE=18.1
DISTRIB_CODENAME=serena
DISTRIB_DESCRIPTION="Linux Mint 18.1 Serena"

And output of snap --version :

max-laptop-linux ~ # snap --version
snap       2.20.1ubuntu1
snapd      2.20.1ubuntu1
series     16
linuxmint  18.1
max-laptop-linux ~ # 

[geekgonecrazy]

Your snap version is even newer then mine. I would think that would be fine.

What do you get from: snap list ? Specifically care about the ubuntu-core version and rocketchat-server version. Just trying to see what doesn't match.

Did you already give removing the snap and re-adding it a shot?

[maxidorius]

max-laptop-linux ~ # snap list
Name               Version  Rev  Developer   Notes
core               16.04.1  714  canonical   -
rocketchat-server  0.48.2   326  rocketchat  devmode
max-laptop-linux ~ # 

Yes I tried to reinstall, reboot, reboot and reinstall, no change.

Looking at the journalctl command output, I see one interesting line (also present in my previous ocmments):
Jan 07 02:11:00 max-laptop-linux snap[4823]: cannot change profile for the next exec call: No such file or directory
Is that expected?

[geekgonecrazy]

@maxidor interesting.. Couple of things.

1. Here is what I have for those:

Name               Version  Rev   Developer   Notes
rocketchat-server  0.48.2   326   rocketchat  -
ubuntu-core        16.04.1  1357  canonical   -

I'll reach out to the snap developers and see if ubuntu-core and core are equivalent. I would assume so.

1. yours says devmode in the notes. How are you starting / installing it?

Never seen that error... but sounds like something related.

[maxidorius]

I'm installing following the instructions on the Downloads page of the website, and this is the output after install:

max-laptop-linux ~ # snap install rocketchat-server
rocketchat-server 0.48.2 from 'rocketchat' installed
max-laptop-linux ~ #

[geekgonecrazy]

@maxidor the fact that its showing devmode leads me to believe there might be something not completely working on linux mint. Because we compile with strict confinement. Only time it would not be strict is if the OS is missing something.

I assume in production you probably wouldn't be installing linux mint. I would suggest trying on regular Ubuntu maybe via a VM. Maybe we can get the Ubuntu guys to take a look. But since it is linux mint... i'm not sure.

[maxidorius]

@geekgonecrazy Works out of the box on a Ubuntu server core 16.04 VM, so it seems the issue is with Mint indeed.
My production system is Debian Jessie so I'll give a go to the docker container now - I'll open a new issue if it doesn't work.
Thank you very much for your support and for the awesome looking product! I'm gonna give it a spin on this Ubuntu VM now.

[geekgonecrazy]

:+1: i'll go ahead and close this. We can revisit this in the future if we need to. But I don't see many production deployments being mint.

[evgenymarkov]

Does anyone have any thoughts about this bug? Mint hasn't snapd by default and after installing it I can't run krita(and probably any other snapped program), it crashes with same error. If you have any experience with debugging snappy subsystem then report a bug to mint bugtracker please.

[geekgonecrazy]

@evgenymarkov from a guess I would say this is something to do with sys calls and apparmor. Its possible you could install the snap with --dangerous and it would work. I'd say the linux mint guys need to take a look though.

[Nikolay-Kha]

Hello, I had the same issue with 'Hello, world!' snap(which was installed with '--dangerous') on Linux Mint 18.1.
This error caused by attemp to write /proc/getpid()/attr/exec which fails with error.
For me, I solved this issue by:
sudo apt-get install selinux
reboot
sudo apt-get install snapd
reboot
these packages conflicts and replaces each other. Somehow, after such reinstall, error disappeared and snap started working for me. Maybe just reinstall of snapd would be sufficient.

[Bruce17]

I have the same error here. Seems the node.js process is not starting properly, because of missing environment variables.

I created my own systemd command as described in https://rocket.chat/docs/installation/manual-installation/pm2-systemd-upstart#how-to-run-rocket-chat-on-systemd-. Now the system is starting up manually.

But I still have issue to autostart, beacause I have to wait in the MongoDB instance. Referencing After= ... snap.rocketchat-server.rocketchat-mongo.service in the custom systemd script seems not to work properly.

My environment is a Ubuntu 16.04 in a LXC container. I installed Rocket.Chat v0.53.0 via snap.

[Bruce17]

Solved it: I installed everything the manual way, now its working

[spr-lfeliu]

Looks like the issue has been fixed on snapd 2.25.
Try running the following command:
snap refresh core --beta
Snappy bug 1687079

[geekgonecrazy]

Let me tack on a caution for anyone that lands here... Be careful with this. If you do this and have other issues with Rocket.Chat please make sure to call out the fact that you are running a different core.

[geekgonecrazy]

If you used the snap as discussed here it's sudo snap install rocketchat-server

[habanany]

I had posted on a wrong threat, thanks though @geekgonecrazy

[ondra-prochazka]

I am experiencing the same issue on Ubuntu 16.04

$ snap list
Name               Version  Rev   Developer   Notes
core               16-2     1689  canonical   -
rocketchat-server  0.56.0   653   rocketchat  -

Log output:

$ sudo journalctl -u snap.rocketchat-server.rocketchat-server
-- Logs begin at Tue 2017-05-30 20:46:38 CEST, end at Wed 2017-05-31 00:48:48 CEST. --
May 31 00:29:30 mobile systemd[1]: Started Service for snap application rocketchat-server.rocketchat-server.
May 31 00:29:30 mobile /usr/bin/snap[28614]: cmd.go:114: DEBUG: not restarting into "/snap/core/current/usr/bin/snap" ([VERSION=2.24 2.24]): older than "/usr/bin/
May 31 00:29:30 mobile snap[28614]: cannot change profile for the next exec call: No such file or directory
May 31 00:29:30 mobile systemd[1]: snap.rocketchat-server.rocketchat-server.service: Main process exited, code=exited, status=1/FAILURE
May 31 00:29:30 mobile systemd[1]: snap.rocketchat-server.rocketchat-server.service: Unit entered failed state.
May 31 00:29:30 mobile systemd[1]: snap.rocketchat-server.rocketchat-server.service: Failed with result 'exit-code'.
May 31 00:29:31 mobile systemd[1]: snap.rocketchat-server.rocketchat-server.service: Service hold-off time over, scheduling restart.
May 31 00:29:31 mobile systemd[1]: Stopped Service for snap application rocketchat-server.rocketchat-server.

etc.

[geekgonecrazy]

Please see the comments in #7000 your error looks related

[ondra-prochazka]

@geekgonecrazy Thanks so much, I am indeed using a custom kernel on my Ubuntu installation.

I cannot change that, but that should not be a problem in production environment where I intend to deploy RocketChat.

[geekgonecrazy]

:+1: yeah unfortunately some of these providers are doing this. I think this is a problem that several VPS providers are doing... :frowning: Might be worth throwing your voice in and asking them to start using an upstream Ubuntu kernel.