Xi-editor: [RFC] User config directory (themes, plugins, preferences)

[cmyr]

Fairly shortly we're going to want to be able to load various user-configurable files, such as themes, plugins, and general settings.

This means there's going to some xi folder somewhere. In line with our general unix spirit, I'm inclined towards the XDG spec. The other alternative would be to respect various platform-specific conventions; e.g. to live in ~/Library/Application Support/ on MacOS? Anyway, call it $XI_DIR.

Some ideas:

• Plugins are in $XI_DIR/plugins, themes in $XI_DIR/themes, etcetera. For themes, we probably cache them somewhere else, and then watch the directory for changes in xi-core, and/or scan on startup. Similarly for plugins, although without the caching.

• General user settings are in $XI_DIR/xi-config.toml or something. Frontends can provide an appropriate UI for general settings (e.g. a preference pane on macOS) but that UI should be reading from / writing to the config file. (does this make sense? Writes might butcher formatting. I'm not sure how good toml is at this.)

• Plugins can provide settings files. A 'defaults' version lives in the plugin package (a directory with some defined structure) and a user copy is stored... somewhere else. Maybe just $XI_DIR/plugin_settings/MyPlugin.toml or something?

Anyway, just wanted to get the ball rolling. Any thoughts?

[trishume]

I mostly agree, but there's two things I don't think I like:

1. I think there should be no distinction between a theme and a plugin. Like Sublime, mostly themes will be a separate type of plugin, but some plugins provide both code and a theme, like PlainTasks. And separating them seems like it will just lead to redundant work. Instead it can be like Sublime where it just loads any theme files found in plugin folders, with auto-reloading.
2. I'm less sure about this, but I would prefer that there was just a default user config directory (overridable with an environment variable like XI_DIR for testing and for NixOS). The user config would have a plugin_search_path setting which would just be an array of paths. The default search path setting would be just the plugins directory next to or under the default user config folder. Then users could add extra paths to accommodate package managers or to keep their plugins and configs separate so they can git version their configs.

An example of why it should be possible to have multiple search paths is for something like http://spacemacs.org/. It has some plugins it installs from a package manager, some it has in its own repo, and a separate directory that users can put their own plugins in that they are working on and don't want to publish or have updated.

[cmyr]

@trishume good thoughts. It definitely makes sense that plugins can include themes. I'm thinking of themes as generally being distinct from plugins because they're a static asset type, but plugins can contain bundled assets. Will think more about that relationship. In any case, this fits with your other observation about using a a path array instead of a path, which I think makes total sense. Behind the scenes theme loading can also use a path array, which will include loading themes from plugins.

[trishume]

I like the Sublime model of packages are just folders of assets of various types, determined by extension. There's themes, keymaps, settings, mousemaps, and for Sublime .py files. For Sublime "package" means a bundle of different assets and "plugin" specifically means a python-based code asset.

The only difference Xi needs to have is that code extension assets are a TOML/YAML/JSON/Whatever file that describes the name and how to launch the plugin, other metadata, and maybe some stuff like scope highlighting priority.

[sp3d]

To what extent is xi-core itself an application, as opposed to an embeddable component? I personally think the latter makes more sense, but I'm not sure that this is the view of the people who are actually doing most of the development. ;) I'm sadly less involved in xi development than most folks in this thread, so I may be underinformed about project vision, but I have concerns about the core managing its own configuration on disk.

In general I would think that embeddable components should get their configuration from the thing that embeds them, i.e. the frontend. Frontends have their own configuration for which they'll independently resolve the issue of where to store things, and frontends may want to modify or be aware of some aspects of xi-core configuration; they are unlikely to be totally ignorant of settings because that would preclude having UI for changing core settings. In addition, I wouldn't expect the core to have its own IPC and instance management to prevent races overwriting its configuration files; I would expect frontends to use platform-specific (dbus on Linux, sockets in /tmp on oldschool unices, WinAPI stuff on that platform, etc.) methods to coordinate opening files in a running instance of the editor, but if the core manipulates its own config files, multiple different frontends (e.g. a terminal-based and a GUI one) on the same machine would fight over them.

Given this, I'd say it's most robust for xi-core to not grub around on the filesystem at all looking for a configuration file. Although users are likely to want some amount of configuration sharing between frontends, if for no other reason than the contemporary computing experience being hopelessly split between the terminal and your favorite GUI realm, that will require coordination on the frontend side in any event. Having xi-core store its own persistent configuration seems like complication we could do without.

[trishume]

@sp3d The concern to balance this against is not wanting to duplicate a bunch of work in the frontend that doesn't have a compelling reason to differ between platforms, as well as being able to port configs between platforms like you mention.

The philosophy is not that xi-core is a library that doesn't do anything platform specific, it is responsible for everything that is easy to do in a cross-platform way and doesn't have a good reason to differ substantially between platforms. For example it does file loading, saving and plugin IPC. File locking is another thing that could be part of the core since it applies to all files not just configs, but probably won't be on the roadmap for a while since it's not a super common problem, many popular text editors don't implement it AFAIK.

Also if by races editing config files, do you mean like one instance changes one setting and a different instance changes a different setting in the same config file, and one of them overwrites the other's change? Because I don't see how locking solves that, solving that (rare) race seems to require a full merge function.

Especially given that users will want their configs to work cross platform, it seems if configuration was part of the frontend it would be mostly duplicated. Every marginal piece of work we add to writing a frontend marginally reduces the number and quality of frontends because of the extra effort required by each frontend author.

To address your concerns about this, we should definitely have an API for accessing and modifying settings, like Sublime text, that the frontends can use to allow customization of their own functionality or to provide nice settings dialogs, but they shouldn't need to bother themselves with file discovery, loading and parsing. There's quite a lot of that work, including parsing and loading plugin configs, syntaxes, themes, key bindings, user settings.

Somewhat related: plugins should also be able to look for resource files in other packages. For example the syntect plugin should be able to look for .sublime-syntax files in all other packages. Sublime Text has an API to do this.

As for embeddability that is something we're thinking about, but more along the line of embedding a front-end. For example this is on the short-term roadmap for using Xi on Fuchsia.

[sp3d]

Basically, I see this as a trichotomy: we can pick one of:

true cooperative sharing of xi-core settings
multiple frontends blindly modifying each other's core settings
no (core-imposed) sharing of settings

The former case requires all running instances of xi-core tied to the same filesystem to know about each other and have some form of IPC between themselves, which they use to broadcast notifications of setting changes to all instances so that settings look coherent from every editor. It means users or frontends may have to switch xi-core settings back and forth if they prefer different core settings with different frontends, but it at least means that the ultimate value of settings won't vary depending on when different instances choose to write settings back to disk.

The second is what you get if you have no coordination but just have each instance of xi-core read its config files when it starts and write them when settings are changed or when it closes. It means you can have an instance think it's persisted its settings to disk when those settings have actually been overwritten on disk by another instance (e.g. from another frontend), and can have multiple frontends whose current views of settings don't agree.

The third is what you'd get by having frontends store their own copies of xi-core configuration and hand them to the core when running it. There's never interference between instances, and the core has to go through the frontend to persist configuration. If multiple frontends do want to establish a protocol for synchronizing settings (a la FF sync, or just local IPC to share between a pair of frontends with similar goals), or a format for exporting them (e.g. a simple "export to file"/"import from file" or such) they can.

You could also look at this from the Rust typesystem angle in terms of Mutex<Config>, UnsafeCell<Config>, and plain old Config.

Configuration of some things will be duplicated between frontends regardless, simply because frontends will have their own settings that the core doesn't know about or which they don't want to share with other frontends (for example, about windowing and application UI). So frontends can't avoid storing or managing configuration entirely even if the core does handle its own configuration.

File locking APIs on most operating systems are basically so broken and foreign to user workflows as to be worthless; I don't see any reason Xi or frontends would care about them.

[trishume]

@sp3d All three of those options are indeed bad when using multiple frontends at the same time. I propose the following solutions:

1. Vastly more people will switch/try out different frontends and want their configs to work between them than people who will actually run multiple frontends at the same time. I basically see no good reason why you would ever want more than one instance of xi-core running on your computer at the same time except being a developer of xi-core. I don't think it's worth optimizing for this use case at the cost of making developing a frontend more complex.
2. Even if you do want to do that, settings files can be handled by the core just like Sublime does: load them on startup, use a file watching interface (there are good cross-platform Rust crates for this) to hot-reload changes, don't save unless you actually make changes (for example don't save on shutdown for no reason). Most concurrent use cases will be solved by this.
3. Frontends can do frontend-specific settings just as their own keys just like Sublime plugins can have their own settings. Through a message to the core like get_setting("xi_mac_show_scroll_bars") and set_setting("xi_mac_show_scroll_bars", true) or some fancier data model than just key-value, like JSON. Thus frontends can avoid creating a settings system while still having frontend-specific settings.
4. Frontends can store state that isn't a setting but should persist between runs, like window layout in open files in their own custom format however they want. The core should indeed not be involved in this. For example in Sublime, window state is stored totally separately. You should be able to version your configs in a Git repo and not have to commit that often, or have anything in there a human can't read.
5. If for some reason you really want to use two different front-ends with different settings concurrently, you can use the mechanism I mentioned in the first comment on this issue and run each frontend/xi-core pair with a different value of XI_DIR and then you basically get the behavior you want, but the core is still doing settings. You can even have each XI_DIR be a git repo on different branches so that you can share and merge changes between them while maintaining differences.