Elvish: Consider following XDG base directory spec

[cesarandreu]

The docs show the default config location is ~/.elvish/rc.elv. Please consider following the XDG base directory spec. The default location would then be ~/.config/elvish/rc.elv.

Moving configs into a folder helps reduce clutter. In the last few years many tools have slowly been adding support: XDG Base Directory support. For comparison, fish shell uses ~/.config/fish/config.fish as its startup file, and lazy-loaded functions go in ~/.config/fish/functions/.

On macOS most tools generally continue using ~/.config as the default location. However, should you choose to follow Apple's guidelines (macOS Library Directory Details), you would use ~/Library/Application Support/elvish/ to place any configs, data, and files. Two other commonly used folders are Caches (~/Library/Caches/elvish/) and Logs (~/Library/Logs/elvish/).

[zchee]

FYI: I wrote xdg basedir package for Go.
https://github.com/zchee/go-xdgbasedir

[fehnomenal]

I want to take a stab on this.

@xiaq are you OK with the following design?

• ~/.elvish/ exists: Do not change behavior because this is most likely a legacy (pre-xdg) install. Output a message on launch to advertise moving to $E:XDG_CONFIG_HOME/elvish.
• Add a flag -config that defaults to $E:XDG_CONFIG_HOME/elvish or ~/.config/elvish.
  dataDir from https://github.com/elves/elvish/blob/702edb3d6523348cb0a8dd44d835432bf11412ab/runtime/runtime.go#L53 gets assigned that flag value.

I would use @zchee's implementation with the Native mode to honor Apple's guidelines, see https://github.com/zchee/go-xdgbasedir/blob/7a0b8922e178cedb342090b90ef662084d88eba8/xdgbasedir.go#L16-L27

[xiaq]

I'm OK with following XDG. However if we are going to move directory, better make it the last directory move we ever have to do. The current .elvish directory has 3 things:

• .elvish/rc.elv: This is configuration.
• .elvish/db: This is user's data.
• .elvish/lib: This is libraries.

XDG has a clear distinction between configuration and data, so clearly rc.elv and db should be in different places. I don't know which bucket .elvish/lib falls into though, the specification does not say. It's worthwhile researching what other languages that respect XDG do.

I'd also prefer that we have a separate implementation in Elvish instead of depending on go-xdgbasedir. I don't mean to discredit their effort, but the main reason is that this behavior needs to be documented in Elvish's documentation, and it's easier to make sure that the behavior and documentation are in sync when they are part of the same codebase.

Elvish also creates temporary files for the socket used for communication with the storage daemon; this is now always /tmp/elvish-$uid. We can use XDG's runtime directory for this, but this can be done separately.

To summarize there are several more things to do:

1. Researching where to put .elvish/lib.
2. Writing Elvish's own implementation for finding the directories.
3. Documenting the behavior in Elvish's documentation.
4. [Optional] Follow XDG for the runtime directory too.

[SolitudeSF]

maybe the lib should be sourced from multiple places, to also account for potential elvish code managed by system package manager and maybe to separate epm fetched libs and user's. something like /usr/share/elvish/ -> $XDG_DATA_HOME/elvish/lib -> $XDG_CONFIG_HOME/elvish/lib. maybe thats overkill.

[xiaq]

@SolitudeSF Yes, I think this is a good opportunity to introduce a system-wide lib too.

On a related note, after reading the XDG spec, it seems that the particular definition it uses for "data files" is not so much as "user's activity data", but more like "assets used in programs", as it cites /usr/share as one of the default paths for data.

In this light, it seems that @fehnomenal's original idea of treating everything as config would make a lot of sense. And since we are back to dealing with just one directory, it makes more sense to implement it ourselves now.

[fehnomenal]

A system-wide lib would be really cool. In preparation to it I will implement the complete XDG Basedir spec. The lib could then be placed somewhere in $XDG_DATA_DIRS.

For the current task I will place

• rc and libs into $XDG_CONFIG_HOME/elvish
• db into $XDG_CACHE_HOME/elvish (it only contains "temporary" data right? I.e. is not configuration)
• the socket into $XDG_RUNTIME_DIR/elvish.sock
• daemon logs into $XDG_DATA_HOME/elvish (debatable but is there a better place?)

[xiaq]

The specification defines cache as "non-essential data", which is pretty vague. But a useful litmus test would be "if cache files are removed, the behavior of the program does not change, except maybe the next startup is slower". This is not true for db.

The XDG base directory specification is written in too vague a way, and it is impossible to make good choices. Treating the entire .elvish directory as config might as well be the least bad one.

[xiaq]

Hmm, after looking at how Fish stores its history in ~/.local/share/fish, I am leaning a bit towards treating db as a data file now.

[krader1961]

The way I think about it is whether the information is unique to a system, in which case it belongs in ~/.local/share/appname, or is part of the apps configuration, in which case it belongs in ~/.config/appname. In other words, is it something I probably want to sync to every system on which I run the app (such as when setting up a new system)? If yes, ~/.config, else ~/.local/share.

[zzamboni]

Call me old-fashioned, but I find XDG confusing and cumbersome, I much prefer plain dot-dirs under $HOME. But I won't object to the change, as I see a lot of other tools following XDG 😐. Also, it's great if a system-wide config dir comes from this 😄

[xiaq]

Right. I am less sure whether the complexity is worthwhile too, especially as the spec doesn't really have good definitions of how to use the different kinds of directories.

The spec is also more complex than I had, and likely most people had, expected. It's a good idea to adopt some standard, but that doesn't equate to adopting any standard. In this particular case, I am no longer sure whether the XDG base directory spec is the correct standard to adopt.

[SolitudeSF]

People expect that XDG is supported. Storing everything in $HOME is incorrect approach.

[ptman]

If one wants to see how the XDG spec can be adapted to different situations, the arch wiki page is quite good: https://wiki.archlinux.org/index.php/XDG_Base_Directory

[Shados]

@xiaq ultimately, adopting a standard that no one else is using is largely equivalent to not adopting a standard. A standard can only functionally be considered a "standard" if it is actually in widespread use, and in this problem space there are really only two candidates that pass that test: XDG, or chuck everything in $HOME/.<something>.

I also agree with @krader1961 that the primary distinction between the use of XDG_DATA_HOME and XDG_CONFIG_HOME is by whether or not data is machine-specific, in practice. To my knowledge, most applications that follow the XDG spec do store "history" files in XDG_DATA_HOME.

[krader1961]

FWIW, the current situation causes problems when ~/.elvish is on a shared filesystem such as NFS. In my case I symlink it to my Dropbox filesystem folder. This causes Dropbox to create copies with names like db (Krader MacBook Pro's conflicted copy 2020-03-11). The important thing here is that the command history database (the db file) is fundamentally local to a machine. Whereas everything else, including the .elvish/lib directory, is almost certainly stuff you want shared with all the machines on which you use elvish.

The XDG standard is not the only way to deal with the above problem. But it's probably the solution with the broadest support.

[krader1961]

If nothing else the situation regarding the location of the Elvish database file needs to be changed. I've been using the rsync command to copy my ~/.elvish directory to the VMs that I use for testing changes. When syncing to Windows 10 that results in this error:

rsync: [receiver] rename "/c/Users/krade/.elvish/.db.T34m17" -> ".elvish/db": Device or resource bus
y (16)

Even on UNIX like systems, where that operation does not result in a fatal error, it is arguably wrong to copy the interactive command database from one system to another when syncing the Elvish config files. Syncing interactive command and location history between systems should be done independent of the Elvish config.

[eternaleye]

@xiaq Regarding the idea of treating everything as config, the rule of thumb I've found best to apply is that it's best if "config" is restricted to some _finite, enumerable_ collection of configurable values, while "data" captures any data that could _not_ be reconstructed simply by enumerating the configurable values, and setting them according to your preferences. This makes it a distinction between "inconvenient to reproduce" and "potentially infeasible to reproduce". Things like history are by nature "data", while by contrast things like "beep/don't beep on tab completion with no candidates" are config.

Shells tend to have a more complicated relationship with this than most GUI applications, as their config is often more nuanced, and itself may be "non-enumerable" due to being dynamically scripted. However, this should (IMO) be read as making it _more like data_, rather than making everything config.

[zzamboni]

@eternaleye I disagree - I think "config" should be anything that is needed to bring the program/application to the settings that you need to use it. In this sense, rc.elv (Elvish), init.el (Emacs), and other code-based configuration files must definitely be considered config, despite their non-enumerable nature.

Another way to put it: config is everything I could not bear to lose (e.g. I would check it into a git repo, possibly share it among machines, etc.), and data is anything that could be eventually reconstructed (e.g. history, command DB, etc.).

[solson]

I agree with @eternaleye's interpretation. I could (with difficulty) reconstruct my shell configuration, but I could never possibly reconstruct the tens of thousands of unique command invocations stored in my shell history. Another example of XDG "data" that might be more illustrative is video game save files, which are infeasible to reproduce. An example of XDG "config" would be the graphics/audio settings inside the game, which are easily reproduced.

Of course, shell config is much more complicated than a bunch of checkboxes and dropdowns, so it hews closer to XDG "data" in terms of how strictly it needs to be preserved (and why I keep mine backed up and in Git), as @eternaleye mentioned. But I just take this as an illustration of what the XDG config/data distinction is intended to mean in general. For a shell, I would expect it to take after Fish and put config/functions in $XDG_CONFIG_HOME/elvish and history or other user data in $XDG_DATA_HOME/elvish.