Neovim: Permissions issue related to ShaDa file after installing from ppa

[Robinlovelace]

I just installed nvim following the instructions on the wiki:

sudo add-apt-repository ppa:neovim-ppa/unstable
sudo apt-get install neovim

• Neovim version:

NVIM 0.1.3
Build type: RelWithDebInfo
Compilation: /usr/bin/x86_64-linux-gnu-gcc -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wconversion -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=1 -O2 -g -DDISABLE_LOG -Wall -Wextra -pedantic -Wno-unused-parameter -Wstrict-prototypes -std=gnu99 -Wvla -fstack-protector-strong -fdiagnostics-color=auto -DINCLUDE_GENERATED_DECLARATIONS -DHAVE_CONFIG_H -D_GNU_SOURCE -I/build/neovim-kQHpaR/neovim-0.1.2ubuntu1+git201604080204+2386+17~ubuntu15.10.1/build/config -I/build/neovim-kQHpaR/neovim-0.1.2ubuntu1+git201604080204+2386+17~ubuntu15.10.1/src -I/build/neovim-kQHpaR/neovim-0.1.2ubuntu1+git201604080204+2386+17~ubuntu15.10.1/.deps/usr/include -I/build/neovim-kQHpaR/neovim-0.1.2ubuntu1+git201604080204+2386+17~ubuntu15.10.1/.deps/usr/include -I/usr/include/luajit-2.0 -I/build/neovim-kQHpaR/neovim-0.1.2ubuntu1+git201604080204+2386+17~ubuntu15.10.1/.deps/usr/include -I/build/neovim-kQHpaR/neovim-0.1.2ubuntu1+git201604080204+2386+17~ubuntu15.10.1/.deps/usr/include -I/build/neovim-kQHpaR/neovim-0.1.2ubuntu1+git201604080204+2386+17~ubuntu15.10.1/.deps/usr/include -I/build/neovim-kQHpaR/neovim-0.1.2ubuntu1+git201604080204+2386+17~ubuntu15.10.1/.deps/usr/include -I/usr/include -I/build/neovim-kQHpaR/neovim-0.1.2ubuntu1+git201604080204+2386+17~ubuntu15.10.1/build/src/nvim/auto -I/build/neovim-kQHpaR/neovim-0.1.2ubuntu1+git201604080204+2386+17~ubuntu15.10.1/build/include
Compiled by buildd@lgw01-52

• [x ] Vim behaves differently?
  
  
  
  • Vim version:
  
  
• Operating system/version: Ubuntu 15.10
• Terminal name/version:
• $TERM:
  

  Actual behaviour

Error message when opening nvim and not root:

E886: System error while opening ShaDa file /home/robin/.local/share/nvim/shada/main.shada: permission denied
Press ENTER or type command to continue

Expected behaviour

Opens without an error message

Steps to reproduce using nvim -u NONE

1. nvim -u NONE

[ZyX-I]

You need to fix permissions of the file in question or directory where it is to be located (or directory where directory is to be located… etc up to the first directory in /home/robin/.local/share/nvim/shada, /home/robin/.local/share/nvim, … which exists). Wrong permissions are not a Neovim issue, Neovim just changed the location of viminfo (and replaced viminfo with ShaDa).

[ZyX-I]

You still can use any other location, check :h shada-n.

[ZyX-I]

You need at least u=rw on file (if it exists) or u=rwx on first directory that exists.

[Robinlovelace]

Agreed - this is an OS setup issue. In the hope that it benefits Linux beginners like me keen on using and supporting NeoVim, this fixed it, thanks @ZyX-I:

sudo chown robin: /home/robin/.local/share/nvim/shada/main.shada

[ZyX-I]

@Robinlovelace You mean you used Neovim when installing your distribution so ShaDa file got user from that other (LiveCD or whatever) system you used to install your distribution from?

Basically if your problem is fixed like this then it means that you first used Neovim from one user, it created ShaDa file there and you now try to write this file from the other user. With this sequence of actions you will get similar failure in Vim.

[Robinlovelace]

@ZyX-I thanks for the insight - learning as I go along. I think that the aim was for future versions to work 'out of the box' and for some reason on Ubuntu and following the instructions on the wiki this was not the case.

[amkgo]

I got this error and find the cause. The error can be shown as following(Ubuntu 14.04):

After install neovim(or sudo rm -R ~/.local), first run sudo nvim filename you will get a .local folder in your home folder with owner root. Next if you run nvim filename you will get E886 error message.

If you first run nvim filename, you will be the owner of the new ~/.local folder. then there will be no problem when you run sudo nvim filename or nvim filename.

I think it's a bug and should always create the ~/.local folder and it's subfolder with the user as owner, not the root.

Andrew

[justinmk]

I think it's a bug and should always create the ~/.local folder and it's subfolder with the user as owner, not the root.

When a command is invoked with sudo, the user is root.

What should touch do when you run sudo touch foo?

[jamessan]

This is a typical problem that users run into when the first time they run (neo)vim on a new system is to edit some privileged file via sudo. As @justinmk says, there's nothing that (neo)vim can do about it.

However, as a user you can learn to use sudoedit instead of sudo $editor.

[Robinlovelace]

So it's my fault for using neovim to edit a txt file as sudo? That seems like a fairly common offence that many others will unwittingly commit! @justinmk and @ZyX-I you'll be happy to hear that I've continued to use nvim even as a unix noob (by nvim developer standards anyway, compared to the everyday public I'm a pro!) and I think it's great software. I just think it should work out-of-the-box. How about a warning message if the user opens it as root on 1st launch? (assuming I've understood the root cause of my issues correctly).

[jamessan]

No one's blaming you. We're just explaining why it happened and suggesting alternatives that would avoid the problem. This exact scenario is when I learned sudoedit. It's a good thing to learn about.

This behavior isn't specific to (neo)vim. Any program which creates files in $HOME will do so as root if you run them via sudo.

[ZyX-I]

@Robinlovelace Using root for running Neovim is perfectly valid. Also sudo is not the only way to use Neovim under root. Sometimes there even may not be any user used to login other then root.

And, actually, for editing files in systems which uses sudo to grant administrative priveleges there is sudoedit and not sudo.

vi editors always assume that user knows what he is doing.

[KillTheMule]

So it's my fault for using neovim to edit a txt file as sudo?

Well, it's not really restricted to neovim. It's basically how sudo operates, namely by keeping most of the environment variables that point neovim to the directories where it stores stuff. This might be considered a feature, and you might argue about that, but at least it's a behavior of sudo, not neovim per se.

You could google up sudo, environment variables and such, but this way lies madness. Try "sudo echo $HOME" (if you can give yourself permissions) to see what it's about and simply accept it :)

[justinmk]

Perhaps we could do a sanity check if HOME does not contain "root".

[justinmk]

(For the case of .local only)

[Robinlovelace]

Great stuff, thanks for the encouraging replies all. One more question from a noob - when I asked the computer to kindly sudoedit neovim /etc/apt/sources.list it rudely behaved not as expected by making this strange beast called 'nano' appear on screen. How rude! Anyway please could you suggest how to set the system editor? Ah actually before I ask that maybe I should check this new invention called the internet!

Trying to stay true to vim culture despite the seductive advertising of Atom. On a total side track, pls could anyone point me towards any recent docs comparing vim/nvim to GitHub/JavaScript ppl's favourite editor?

I think usability and user friendliness is absolutely key to the uptake and community building of software so look forward to updates in that direction now that many of the core objectives of the awesome nvim project seem on their way to completion.

[jamessan]

when it asked the computer to kindly sudoedit neovim /etc/apt/sources.list

The command is just sudoedit /etc/apt/sources.list. sudoedit checks a few environment variables to determine which editor to use and when those aren't present (which is likely the case), falls back to using /usr/bin/editor.

it rudely behaved not as expected by making this strange beast called 'nano' appear on screen.

On Debian based systems, you can configure which tool provides the /usr/bin/editor symlink using the update-alternatives command -- sudo update-alternatives --config editor.

---

Ah, I should've checked your link before responding. :)

[ZyX-I]

You could google up sudo, environment variables and such, but this way lies madness. Try "sudo echo $HOME" (if you can give yourself permissions) to see what it's about and simply accept it :)

sudo echo $HOME will expand $HOME _before_ even running sudo thus making it equivalent to echo $HOME, so it is useless to run. Use sudo zsh -c 'echo $HOME' if needed, or something like sudo env | grep HOME.

[KillTheMule]

Hmm you're right of course, but doesn't that mean that nvim should write to '/root/.local' indeed when used in sudo nvim?

[jamessan]

No, because $HOME is still going to reflect the user's $HOME due to how sudo works. ZyX-I was just critiquing the methodology not suggesting that the result would be different.

[KillTheMule]

When I tried what he suggested, /root was returned, so at least in my case it did not reflect the user's home. Sorry if I did not make that clear :)

(e) Oh ok, from the manpage:

 -H, --set-home
             Request that the security policy set the HOME environment
             variable to the home directory specified by the target user's
             password database entry.  **Depending on the policy, this may
             be the default behavior**.

Emphasis mine.

[justinmk]

@KillTheMule sudo zsh -c 'echo $HOME' is intended to demonstrate that un-expanded $HOME still inherits the parent env when it is eventually expanded by the root-owned zsh process. With sudo echo $HOME, the user-owned shell expands it before executing echo, whereas with sudo zsh -c 'echo $HOME', the single-quotes prevent expansion by the user-owned shell: the string echo $HOME is sent literally to the child zsh shell.

[KillTheMule]

@justinmk I understood, see above. It's just that sudo bash -c 'echo $HOME' _does_ return '/root' for me. Same for sudo env|grep HOME.

[fwalch]

This seems to be due to distro defaults. Arch Linux sets env_reset in sudoers by default, thus $HOME is /root under sudo. On Ubuntu, $HOME is preserved under sudo.

[amkgo]

So may be some explaination in the documents/FAQ will be great and the error message can give some hints about this or link to the document will be better.

[jamessan]

At least in neovim you get an indication this is happening from the error. :) In Vim, the viminfo file is just silently not updated.

[amkgo]

Is there any chance to let neovim check ~/.local when it's used first time or just after installation?

[ZyX-I]

Check what?

[amkgo]

Check the current user name(like command whoami) when nvim starts running and create the shada file in that user's home folder:
/home/username/.local/share/nvim/shada/main.shada

If I use sudo nvim filename it will be /home/root/ folder, if only nvim filename it will be /home/andrew.

[jamessan]

If I use sudo nvim filename it will be /home/root/ folder

That depends on how you have sudo configured. Some distributions have sudo's default configuration set $HOME based on the target user, while others preserve $HOME of the calling user. There are reasons to do both.

Yours appears to be in the latter camp, and that's violating your expectations, so why not change the behavior of sudo to match your expectations (if you don't think that using sudoedit is a valid alternative)? (Neo)vim isn't the only tool that's going to honor $HOME when run under sudo.

[amkgo]

Lots of people like me are expected to install neovim and run immediately. They may has little or no Linux administrator experience.

If sudo's $HOME is /home/root/, great, this is as expected.

If it's /home/andrew/ like my case, there are two situation:

1. The ownership of /home/andrew./local/share/nvim/shada/main.shada is the calling user - andrew - great, this is as expected too.
2. But if the ownership is root, this is my case, we can either report to the user to change owership when E886 shows, or simply let neovim create an alternate file and use that file instead when it's not root.

It's like vim will find the config file in several places. Neovim can choose the shada file in an alternate place. This way no error shows, user with no administrator knowledge will just use it.

[Robinlovelace]

Heads up - just installed nvim on a new computer with Ubuntu 16.04 and I'm getting the same error, after following the Wiki. @justinmk I think this issue should be re-opened as _neovim does not work out-of-the-box following the recommended installation method on the wiki_. I'm sure this issue could be resolved quite simply but not quite sure how.

[Robinlovelace]

sudo chmod -R 777  ~/.local/share/nvim

That fixed it this time (the previous chmod command didn't...).

The ownership of ~/.local/share/nvim should be the user's by default on Ubuntu, I'm sure of it!

[justinmk]

@Robinlovelace Just to be clear, the exact, sufficient steps to reproduce this are:

sudo add-apt-repository ppa:neovim-ppa/unstable
sudo apt-get install neovim
nvim

Correct? If so, then I guess the problem is that nvim is invoked by the install step, without -i NONE (perhaps to generate helptags--does that sound correct @fwalch ?). Maybe we could prevent this issue by specifying -i NONE in all installation steps?

[jamessan]

I guess the problem is that nvim is invoked by the install step, without -i NONE (perhaps to generated helptags--does that sound correct @fwalch ?)

I don't see anything in the packaging repo that would be doing that. Downloading the deb itself and unpacking the control files, I can confirm that there are no maintainer scripts that get run as part of the install.

It sounds like there's an intervening step between sudo apt-get install neovim and nvim that is causing this, which likely looks something like sudo nvim ....

[justinmk]

We could append .1 to the default shada path if user doesn't have permissions to the existing one.

[Robinlovelace]

@justinmk in response to your question: yes that will reproduce it on most people's Ubuntu systems I do believe. That's what I did in any case. Thanks for re-opening and not blasting me for using Ubuntu!

[Robinlovelace]

If someone with an Ubuntu machine could reproduce, that would be good...

[fwalch]

Here is a Dockerfile (based on this; save as Dockerfile, run sudo docker build --no-cache . in that directory).

FROM ubuntu:16.04

RUN DEBIAN_FRONTEND=noninteractive; apt-get update; apt-get install --no-install-recommends -y software-properties-common; add-apt-repository ppa:neovim-ppa/unstable; apt-get update; apt-get upgrade; apt-get install --no-install-recommends -y neovim; apt-get clean; rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*

WORKDIR /home/nvimuser
ENV HOME /home/nvimuser

RUN adduser --disabled-password -u 500 nvimuser
RUN chown -R nvimuser:nvimuser /home/nvimuser
USER nvimuser

RUN nvim --headless -c 'qall'
RUN ls -l $HOME/.local/share/nvim/shada/main.shada

And it outputs:

Step 10 : RUN ls -l $HOME/.local/share/nvim/shada/main.shada
 ---> Running in b7d68d3deb9a
-rw------- 1 nvimuser nvimuser 86 Aug 29 19:53 /home/nvimuser/.local/share/nvim/shada/main.shada

So, the shada file is created with correct permissions.

[justinmk]

@Robinlovelace Are you able to reproduce this without your personal config? Perhaps you could narrow down the config differences that are causing this for you. Until then we don't have steps to reproduce.

[Robinlovelace]

Hi Justin, not sure what I should to reproduce this - I can uninstall and re-install it... Not quite sure sure whould you mean by "this without your personal config".

[fwalch]

Not quite sure sure whould you mean by "this without your personal config".

As you can see from the Dockerfile, the shada file is correctly created if you run just the steps outlined in https://github.com/neovim/neovim/issues/4551#issuecomment-243186355 on a clean Ubuntu system. The only (?) thing that could still make a difference is an init.vim or some plugins ("personal config").

We could append .1 to the default shada path if user doesn't have permissions to the existing one.

To make this more clear, we could create the .shada files as $HOME/.../$USER.shada. That would at least help to see which user a shada file belongs to.

[Robinlovelace]

Thanks for the clarification. Let me know if there's anything else I can do to help diagnose the issue on my system (e.g. uninstall neovim and re-install it with different settings). As I say it's a relatively new Ubuntu 16.04 install so suspect it will affect other Ubuntu users.

[fwalch]

Something like https://github.com/fwalch/neovim/commit/54201f893fe01df7a27e6f959f92e59c19a53f0e should work, I guess, although I am not sure about this solution.. if sudo nvim is the first invocation, the directories will still be owned by root.

[jamessan]

Let me know if there's anything else I can do to help diagnose the issue on my system (e.g. uninstall neovim and re-install it with different settings).

You still haven't given a complete set of steps to reproduce, from a clean install. @fwalch already showed using Docker that the steps you provided don't exhibit the problem. As I stated previously, the scenario you're describing is likely happening because you're running nvim as root (via sudo) before you run it as your own user.

We've already described why that's not a good idea and that using sudoedit instead both avoids the issue and is a generally safer approach (fewer things running as root). Because of the sudo configuration, $HOME is preserved so running sudo nvim ... causes the root user to create the directory. While this is annoying, it's at least discoverable. Neovim emits an error describing the problem and you can trivially fix it by removing the directory. Plus, it gave you an opportunity to learn something new.

This isn't specific to Neovim. It's a combination of the sudo configuration and the command that you're running needing to create something in $HOME. This isn't something that can be easily fixed because it requires making assumptions about what the user wants rather than having predictable behavior. Therefore, I'm closing this.

[Robinlovelace]

Ah yes that would explain it - because I tend to use vim for modifying system files the first usage was likely

sudo nvim /etc/apt/sources.list

Apologies for any hassle, and thanks for being so on it.

[kfrncs]

sudo chmod -R 777  ~/.local/share/nvim

That fixed it this time (the previous chmod command didn't...).

The ownership of ~/.local/share/nvim should be the user's by default on Ubuntu, I'm sure of it!

This is still happening on my Raspberry Pi with Ubuntu Mate. Fresh install, might have done sudo nvim on a read-only file as my first open. This command fixed it.

Distributor ID: Ubuntu, Description: Ubuntu 16.04.2 LTS, Release: 16.04, Codename: xenial