Borg: Why does XDG_CONFIG_HOME have precedence over BORG_BASE_DIR?

[finefoot]

I want to store the complete borg app data (config, cache, etc.) together with the backup data (in a different folder) on my external hard drive. The docs at https://borgbackup.readthedocs.io/en/stable/usage/general.html#environment-variables say:

BORG_BASE_DIR
Default to $HOME, ~$USER, ~ (in that order). If we refer to ~ below, we in fact mean BORG_BASE_DIR.
BORG_CONFIG_DIR
Default to ‘~/.config/borg’. This directory contains the whole config directories.

This appears to imply that I won't have to set BORG_CONFIG_DIR, BORG_CACHE_DIR etc. separately but all is done with setting BORG_BASE_DIR. (In my view, this is not only convenient but actually makes sense in case another path variable will be added in the future for some feature xyz like ~/.xyz/borg with BORG_XYZ_DIR, which I wouldn't have to explicitly set since I have already set BORG_BASE_DIR.)

However, this does not work if XDG_CONFIG_HOME is set, as it has precedence over BORG_BASE_DIR when the other paths are being assembled. What's the reason behind this?

I'm currently using borg-linux64 1.1.10, but the issue is relevant to both master and 1.1-maint: See https://github.com/borgbackup/borg/blob/master/src/borg/helpers/fs.py#L80 and https://github.com/borgbackup/borg/blob/1.1-maint/src/borg/helpers.py#L567, respectively.

I would have expected something like this

config_home = os.path.join(get_base_dir(), '.config')
if not os.environ.get('BORG_BASE_DIR'):
    config_home = os.environ.get('XDG_CONFIG_HOME', config_home)
config_dir = os.environ.get('BORG_CONFIG_DIR', os.path.join(config_home, 'borg'))

or this

config_dir = os.environ.get('BORG_CONFIG_DIR')
if not config_dir:
    config_home = os.path.join(get_base_dir(), '.config')
    if not os.environ.get('BORG_BASE_DIR'):
        config_home = os.environ.get('XDG_CONFIG_HOME', config_home)
    config_dir = os.path.join(config_home, 'borg')

as well as XDG_CONFIG_HOME being mentioned in the docs.

[ThomasWaldmann]

The cache data is not encrypted, so if you have your repo encrypted (as it is by default), it is not a good idea to store the cache files with the repo files.

[finefoot]

The cache data is not encrypted, so if you have your repo encrypted (as it is by default), it is not a good idea to store the cache files with the repo files.

Sorry for the misunderstanding: with "backup data" I was speaking of the data which will be backed up. The encrypted borg repo is not on that external hard drive.

[ThomasWaldmann]

Currently the code is so that XDG_(CONFIG|CACHE)_HOME overrides get_base_dir result (which internally uses BORG_BASE_DIR, HOME, ~$USER).

I have to admit that I do not know what the "usual" behaviour of these should be.

There is that:
https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html

But how it relates to application specific env vars is basically undefined.

[finefoot]

Currently the code is so that XDG_(CONFIG|CACHE)_HOME overrides get_base_dir result (which internally uses BORG_BASE_DIR, HOME, ~$USER).

Yes, I have had a look at the code and it's pretty straight forward. However, I only did have a look after I couldn't work out why borg didn't work like I expected: There is no mentioning of XDG_CONFIG_HOME or XDG_CACHE_HOME in the docs at https://borgbackup.readthedocs.io/en/stable/usage/general.html#environment-variables So I had only set BORG_BASE_DIR and wondered why nothing changed. (Because borg used XDG_CONFIG_HOME and ignored BORG_BASE_DIR.)

But how it relates to application specific env vars is basically undefined.

In my view, XDG_CONFIG_HOME and XDG_CACHE_HOME are "generic" variables which are being set for multiple applications, not only for borg. At the same time, setting BORG_BASE_DIR is borg-specific. Why then should borg ignore the borg-specific value from BORG_BASE_DIR if it has explicitly been set by the user and use the generic one instead?

To me it would make more sense to use the following order from highest to lowest precedence:

1. BORG_CONFIG_DIR or BORG_CACHE_DIR if it has been set
2. BORG_BASE_DIR if it has been set
3. XDG_CONFIG_HOME or XDG_CACHE_HOME if it has been set
4. get_base_dir

See my original post above for the code equivalent example for get_config_dir. Could be applied to get_cache_dir analogously.

[ThomasWaldmann]

Sounds good! :-)

Guess we need:

• a PR against master code and docs (implementing better behaviour, describing behaviour / updating env var docs)
• after that: thoughts about whether to apply against 1.1-maint or not
• either update 1.1-maint to describe current behaviour better or update code/docs to new "fixed" behaviour

[finefoot]

a PR against master code and docs (implementing better behaviour, describing behaviour / updating env var docs)

Which one of the two code variants do you like better? They differ semantically but effectively do the same, so it comes down to keeping borg's code style consistent.

Furthermore, it seems sensible to wait with docs additions until #4900 is taken care of to avoid possible merge conflicts, right?

after that: thoughts about whether to apply against 1.1-maint or not

In the unlikely case that someone is currently using XDG_CONFIG_HOME or XDG_CACHE_HOME while at the same time having BORG_BASE_DIR set (even if it doesn't have an effect), the "new" behavior (of giving precedence to BORG_BASE_DIR) could potentially "break" their setup if the given locations don't match. However, the worst case is a cache rebuild (for changed cache location) and a notification that the archive seems to have been altered (for changed config location) if I'm not mistaken?

[ThomasWaldmann]

I think it is good to start with a PR against master, implementing fixed behaviour and updating the docs.

After that is finished, we can think about whether the fix is ok for 1.1-maint also. Maybe it is not as critical, as you have told above.

[ThomasWaldmann]

Guess we could do a backport to 1.1-maint.

[ThomasWaldmann]

master: fixed by #5066