Hledger: Please follow the XDG Base Dir Spec

[billop]

Although hledger allows specifying any file for configuration, it would be nice to have sane defaults that follow the spec and falls back to the legacy default locations.

There is a nice haskell library that implements this.

[simonmichael]

Thanks @BilLOPGVkPPn8z0JGJhgiAGNo9YAqpk5ruChl0F . Would you care to sketch out a design ? Eg what would change in hledger, and how would we handle the transition.

[simonmichael]

Also, how would you describe the impact/cost of the current behaviour ? Since we don't currently claim to follow that spec, perhaps this is more of a wish than a bug ? Ie how urgent/important is it.

[billop]

Eg what would change in hledger, and how would we handle the transition.

XDG Base Dir env vars should be used if present, if not, fall back to the original defaults. Note that ~/.config, ~/.local/share, etc. would be better fallbacks and any sort of home pollution should be a last resort, but this requires breaking backwards compatability.

I'm hesitant to be more concrete as I'm not familiar with the internals of this project (and Haskell), but it seems that the ledger file should be searched for at "$XDG_DATA_HOME"/hledger.journal by default. Since hledger-ui for example does not create the file if missing, it should just be a single if statement (if that's idiomatic Haskell) to check if the relevant XDG Base Dir env vars are present, and if they are not, behave exactly the way hledger behaves right now (unless full compliance to the spec is desired).

Also, how would you describe the impact/cost of the current behaviour ?

The cost of the current behavior is home pollution and not following the freedesktop.org spec, which among other aspects of Linux as a desktop OS, standardizes how applications should interact with the filesystem for configuration, caching, storing data, etc.

Since we don't currently claim to follow that spec, perhaps this is more of a wish than a bug ?

I thought "bug" was a more appropriate label since any application that supports Linux released after 2000 should follow the standard agreed upon by all major players in the Linux desktop world, but I think it's really up to how much the maintainers/users care. If it is a bug I would say it's a minor one.

Ie how urgent/important is it.

Since no one else in the plaintext accounting ecosystem has ever raised such an issue, I would say that urgency and importance is low. An issue however is that the longer it's not followed, the more difficult it becomes to follow in the future.

[simonmichael]

Thanks for the input. I'm probably in favour of moving in this direction, especially if someone wants to work on it. This has potentially a lot of details to be worked through. I'm certainly interested in exploring it further.

Let me cherry-pick one issue, the default location of the journal file.

• There is a default location so that new users can run "hledger add" and then the other hledger commands without needing any configuration.
• I picked the home directory for simplicity and also because your financial journal is important data and I didn't want it getting lost or accidentally deleted from being deep in some hidden dot directory.
• I gave it a dotted name (.hledger.journal) so that it wouldn't be so obvious as to be annoying when you're not using it. Now I think I should have chosen a non-dotted name - if it's important data then obvious is good.
• I still have doubts about storing this valuable file in a hidden subdirectory, where users might have trouble finding it or preserving it. hledger aims to be useful also to non-technical users.
• On windows, the user's home directory moves around, based on OS version and other things. I wouldn't want to do anything that makes this file harder to find on windows.

To flesh this out, here are the two use cases I've been thinking of:

1. A new user starts using hledger via add/iadd commands, following tutorials, and never thinks about where the data goes. Some folks will skip this step.
2. Later on they want to use version control, or multiple files (included journals, csv files, csv rules..) which are best kept together, or they want to use an encrypted folder, or whatever. At this point they move the default file ($HOME/.hledger.journal) to its new location, and they deal with configuring the LEDGER_FILE environment variable (in all the environments where it's needed; or setting up command aliases to achieve the same effect).

[billop]

Thanks so much for looking in this, the biggest disappointment with the plaintext accounting ecosystem has been how every tool seems to just dump everything into my home folder, I'm glad to see serious consideration.

I see what you mean about keeping the journal file location simple. However, with what I've proposed (leaning towards legacy fallback to be pragmatic) I don't think that's an issue at all. If there are no XDG base dir env vars present, nothing would change. Unless I'm mistaken, these env vars must be set manually, so a user that does not care would not have these env vars. I would be very happy with this behavior, even if it didn't fall back to ~/.local/share and all that.

So in terms of the two use cases you mentioned:

1. New users that do not care about the location - nothing changes
2. Users that are more involved with the files - setting yet another env var becomes unnecessary

[simonmichael]

I don't quite follow, if/when you have some time could you spell out the functional specification ?

[billop]

could you spell out the functional specification ?

Do you mean the XDG spec?

https://wiki.archlinux.org/index.php/XDG_Base_Directory

[simonmichael]

I mean a specification for how hledger should behave (in detail).

[simonmichael]

If it's easier, we could chat about this on the IRC channel too.

[simonmichael]

Some related resources:
https://wiki.archlinux.org/index.php/XDG_Base_Directory
https://en.wikipedia.org/wiki/Hidden_file_and_hidden_directory
https://www.freedesktop.org/wiki/Specifications/basedir-spec/
https://specifications.freedesktop.org/basedir-spec/latest/
https://wiki.debian.org/XDGBaseDirectorySpecification
https://developer.gnome.org/basedir-spec/

[simonmichael]

Here are hledger's current interactions with the filesystem, IIRC:

• Main input file (usually journal format, but need not be)
  
  
  
  • if LEDGER_FILE is set, we use that; otherwise we use $HOME/.hledger.journal (or Windows equivalent)
  
  
  • most commands try to read this file
  
  
  • a few commands try to write there (add, import, maybe others - iadd, ui, web ?)
  
  
• Included files
  
  
  
  • Journal files can reference other files, in journal or other formats, to include additional transactions or directives
  
  
• Argument files
  
  
  
  • An argument like @FILE uses FILE's contents as command line arguments
  
  
• No config file
  
  
  
  • hledger currently has no special config file (though you can emulate one with the features above)
  
  
• CSV rules files
  
  
  
  • When reading any CSVFILE, hledger also tries to read CSVFILE.rules in CSVFILE's directory, or another specified by --rules-file
  
  
• Import history files
  
  
  
  • For each FILE read by the import command, it reads and writes .latest.FILE in FILE's directory
  
  
• hledger-web session file
  
  
  
  • hledger-web writes a .hledger-web_client_session_key.aes file in the current directory at startup
  
  

[simonmichael]

Hypotheses:

• some hledger users get started by running hledger add (following the docs) and then just using hledger, without caring where the journal file is
• the majority of users will sooner or later move that file to a dedicated directory, eg to keep multiple files organised, to keep files in version control, or to facilitate backups/encryption/sharing. Usually this also means configuring LEDGER_FILE, or a command alias, to avoid typing -f FILE every time.

[simonmichael]

• configuring LEDGER_FILE or other environment variables is something of a hassle*, and reducing the need for that reduces friction for users. (* How do you configure LEDGER_FILE so that it sticks ? So that it takes effect now ? And in other terminals you had open ? What about when you start hledger tools from inside emacs ? from the GUI ? on Windows ? How do you document it for people not using bash ?)

[simonmichael]

Sorry, I left this too long. Here are the thoughts I remember:

Re moving hledger's default journal to $HOME/.local/share/hledger/hledger.journal: I think it would be a disimprovement for the majority of hledger users using this file. That path requires remembering four things rather than one ($HOME/.hledger.journal). Also it suggests data that is owned and managed by the application, like say Firefox's bookmarks database, but hledger's data isn't really hledger's; it is primarily managed directly by the user, using multiple tools, version control etc. It should be in the place most convenient for them. I'm against this for now. We could review in a year (perhaps the BDS will improve this, or become more universally adopted).

Re the .hledger-web_client_session_key.aes: we should fix it not to write that, since we don't currently need sessions. Otherwise, we should make hledger-web store it in ~/.config/hledger/ $XDG_RUNTIME_DIR (which has no default, we'd have to pick it). Though, that would need further thought about what to do when the user runs multiple hledger-web instances.

If we ever add an official config file, it could be ~/.config/hledger/hledger.conf.

Re all the other kinds of files mentioned above, like the journal file I think they are under the user's control or belong next to their associated data files, so no change is needed for these.

[billop]

Not sure if session keys should be in ~/.config, but overall seems very reasonable.

[alerque]

My opinion is that hlegder needs to give up having a "default" location at all and always operate on some file specified. Financial data is mine, not my app's nor just something it puts somewhere for me. Of all the things I have that needs good data management (versioning, backup) financial records are high in the list. It is not equivalent to a dotfile. Stop pretending it is. Fix the convenience issue another way. Perhaps have a default filename relative to PWD but no more.

[simonmichael]

Not sure if session keys should be in ~/.config, but overall seems very reasonable.

Cool. You're right, I meant $XDG_RUNTIME_DIR for that one. I see that has no default value, so we'd have to pick one.

@alerque, I agree. As I tried to say above, the default journal location is mainly just a convenience to get less-technical new users started easily. We expect that power users will actively manage their journal, somewhere else, and will set the LEDGER_FILE environment variable.

Perhaps we should open a new issue to continue thinking about this. Another option could be: when LEDGER_FILE is unset, assume a standard filename in the current directory.

[billop]

XDG_RUNTIME_DIR is set by default through pam_systemd

https://wiki.archlinux.org/index.php/XDG_Base_Directory#Specification