Kibana: Upgrade causes root owned files

[markwalkom]

And fails when (re)start(ing) KB after the upgrade;

fs.js:565
fs.write = function(fd, buffer, offset, length, position, callback) {
                                                 ^
Error: EACCES, permission denied '/opt/kibana/optimize/.babelcache.json'
    at Error (native)
    at Object.fs.openSync (fs.js:500:18)
    at Object.fs.writeFileSync (fs.js:1099:15)
    at save (/opt/kibana/node_modules/babel-core/lib/api/register/cache.js:35:19)
    at process._tickDomainCallback (node.js:381:11)
    at Function.Module.runMain (module.js:503:11)
    at startup (node.js:129:16)
    at node.js:814:3
FATAL { [Error: EACCES: permission denied, open '/opt/kibana/optimize/bundles/marvel.entry.js']
  cause:
   { [Error: EACCES: permission denied, open '/opt/kibana/optimize/bundles/marvel.entry.js']
     errno: -13,
     code: 'EACCES',
     syscall: 'open',
     path: '/opt/kibana/optimize/bundles/marvel.entry.js' },
  isOperational: true,
  errno: -13,
  code: 'EACCES',
  syscall: 'open',
  path: '/opt/kibana/optimize/bundles/marvel.entry.js' }

Here are the files;

# ll /opt/kibana/optimize/.babelcache.json
-rw-r--r-- 1 kibana root 89511 Apr  1 04:41 /opt/kibana/optimize/.babelcache.json
# ll /opt/kibana/optimize/bundles/marvel.entry.js
-rw-r--r-- 1 root root 3663 Mar 19 23:17 /opt/kibana/optimize/bundles/marvel.entry.js

I did a dirty chown -R kibana /opt/kibana/, but not sure if there is a cleaner settings.

[epixa]

@spalger While we can't fix the permissions on the file, we could potentially have kibana write the babel cache file to a tmp directory instead. What do you think?

[mikeh-elastic]

This permissions issue also affects the rpm of kibana: kibana-4.5.0-1.x86_64

[spalger]

The optimize directory is configurable, but I don't think a temp directory is a good idea. We don't want these files to ever be wiped out by the OS (babel cache is only one of the files that gets written)

[LeeDr]

I created another issue https://github.com/elastic/kibana/issues/7157 that plugins need a dir (owned by Kibana) they can write to.

[lifeofguenter]

Similar issue with phantomjs (x-pack):

[10:55] <w33ble> the fix for now is to manually adjust the permissions. most likely you just need write permissions on /opt/kibana/installedPlugins/x-pack/.phantom
[10:56] <w33ble> 6730 is somewhat related
[10:56] <lifeofguenter_> ah cool, so I only need to chmod/chown that subdirectory and kibana will pick it up the next time it restarts?
[10:56] <lifeofguenter_> e.g. I don't need to re-install the xpack or so?
[10:57] <w33ble> nah. the issue is with uncompressing the phantom package, so that's the only part you need to allow
[10:57] <w33ble> in fact, once you run it the first time, you can put the permissions back
[10:57] <w33ble> or, you can unpack by hand

[LeeDr]

@lifeofguenter I think https://github.com/elastic/x-plugins/issues/2201 covers that issue.

[lifeofguenter]

Thanks @LeeDr , I don't think I have access to that repository, hence why I did not find it in first place :)

[markwalkom]

https://discuss.elastic.co/t/kibana-4-5-1-cannot-start-because-of-permission-denied/50392 seems related

[ppf2]

Same thing with 5.0 Alpha 3 (even for a fresh install). Install 5.0 Alpha 3, install x-pack, the /opt/kibana/optimize/.babelcache.json is root owned and the kibana service will not start up until it is changed to kibana owner.

[epixa]

@jbudz Does that PR also solve the problems for files other than babelcache? Reopening this just until confirmation.

[jeremyforan]

i resolved it by change the service user:

vim /etc/default/kibana

from:
user="kabana" group="root" chroot="/" chdir="/" nice=""

to:
user="root" group="root" chroot="/" chdir="/" nice=""

Not really sure if this is the ideal solution. Running Centos

[epixa]

@jeremyforan Thanks for that. I think this will be fixed from 4.6.0 onward.

[fredvd]

This issue is still open for everybody who tries to upgrade an existing kibana 4.x to 4.5 on Ubuntu. Found this ticket after stumbling and prodding for an hour :-( 👎

[epixa]

@fredvd Unfortunately this is not something that we can retroactively fix since the underlying permissions issue is the result of the circumstances at the original install time rather than at upgrade time. So when the fixed version is released, it will require a fresh install, and then all upgrades from that point onward should work as you'd expect.

It's possible the fix will go out in 4.5.4 instead of 4.6.0, but we're not really sure yet.

[celesteking]

fresh install, same problem.

$ id
uid=995(kibana) gid=993(kibana) groups=993(kibana)
$ /usr/share/kibana/bin/kibana -c /etc/kibana/kibana.yml 
  log   [13:49:36.633] [fatal] Error: EACCES: permission denied, open '/usr/share/kibana/optimize/bundles/monitoring.entry.js'
    at Error (native)                                                                                                                                                                                                                        
FATAL { Error: EACCES: permission denied, open '/usr/share/kibana/optimize/bundles/monitoring.entry.js'
    at Error (native)
  cause: 
   { Error: EACCES: permission denied, open '/usr/share/kibana/optimize/bundles/monitoring.entry.js'
       at Error (native)
     errno: -13,
     code: 'EACCES',
     syscall: 'open',
     path: '/usr/share/kibana/optimize/bundles/monitoring.entry.js' },
  isOperational: true,
  errno: -13,
  code: 'EACCES',
  syscall: 'open',
  path: '/usr/share/kibana/optimize/bundles/monitoring.entry.js' }

RPM kibana-5.0.2-1.x86_64

update: This occurs when disabling x-pack components, as per guide.
chown -R kibana /usr/share/kibana/optimize/ fixed the problem.

[jbudz]

@celesteking tracking that one at https://github.com/elastic/kibana/issues/8818

[niemyjski]

I just ran into this exact same error with 5.1.2, what I did was edited the kibana.yml file to disable non free xpack plugins. When I restarted the node, it wouldn't boot.

[rodgermoore]

Still run into this issue on 5.3. Can we fix this please? Tnx

Edit: I posted this in the wrong issue. I ran into this error after installing x-pack on Kibana 5.3.

[emanzaman]

@spalger : Could you please let me know how to configure optimize directory?

[spalger]

@emanzaman I assume that since you commented here that you know the optimize directory can't be configured...

[emanzaman]

@spalger Thanks for the confirmation but when we copied the whole optimize directory to a different folder (owned by kibana user) and tried exporting BABEL_CACHE_PATH which is the path upto ${DIR_OWNED_BY_KIBANA_USER}/optimize/.babelcache.json file, kibana seems to be working fine. Is there any other problem that we might encounter in future?
PS: We are running kibana 5.6.8
Thanks.

[Aeres-u99]

I am on Version 7.8.1

vagrant@vagrant:/usr/share/kibana/bin$ ./kibana --version
7.8.1
Babel could not write cache to file: /usr/share/kibana/optimize/.babel_register_cache.json 
due to a permission issue. Cache is disabled.

Error still persists,

vagrant@vagrant:/usr/share/kibana/bin$ ./kibana
Babel could not write cache to file: /usr/share/kibana/optimize/.babel_register_cache.json 
due to a permission issue. Cache is disabled.
FATAL CLI ERROR Error: ENOENT: no such file or directory, open '/usr/share/kibana/config/kibana.yml'
    at Object.openSync (fs.js:443:3)
    at readFileSync (fs.js:343:35)
    at readYaml (/usr/share/kibana/src/core/server/config/read_config.js:34:69)
    at getConfigFromFiles (/usr/share/kibana/src/core/server/config/read_config.js:66:18)
    at RawConfigService.loadConfig (/usr/share/kibana/src/core/server/config/raw_config_service.js:51:70)
    at bootstrap (/usr/share/kibana/src/core/server/bootstrap.js:61:20)
    at Command.<anonymous> (/usr/share/kibana/src/cli/serve/serve.js:195:33)
    at Command.<anonymous> (/usr/share/kibana/src/cli/command.js:111:20)
    at Command.listener (/usr/share/kibana/node_modules/commander/index.js:291:8)
    at Command.emit (events.js:198:13)
    at Command.parseArgs (/usr/share/kibana/node_modules/commander/index.js:672:12)
    at Command.parse (/usr/share/kibana/node_modules/commander/index.js:459:21)
    at Object.<anonymous> (/usr/share/kibana/src/cli/cli.js:60:9)
    at Module._compile (internal/modules/cjs/loader.js:778:30)
    at Module._compile (/usr/share/kibana/node_modules/pirates/lib/index.js:99:24)
    at Module._extensions..js (internal/modules/cjs/loader.js:789:10)
vagrant@vagrant:/usr/share/kibana/bin$ 

What all I have tried:

1. @jeremyforan 's solution of changing service user to root
2. Chowning /usr/share/kibana and /usr/share/kibana/optimize recursively to kibana user.
3. Sudo ./kibana will work with --allow-root but I am afraid that's not what I want.

EDIT:
Furthermore, Kibana was installed using the deb package on ubuntu 20.04

[gpavinteractiv]

Same problem since upgrade from to 7.8.1 to 7.9 on centos 7