Borg: BORG_PASSCOMMAND example in FAQ wrong

[reyman]

Have you checked borgbackup docs, FAQ, and open Github issues?

Yes

Is this a BUG / ISSUE report or a QUESTION?

Bug

System information. For client/server mode post info for both machines.

Your borg version (borg -V).

1.1.8

Operating system (distribution) and version.

Ubuntu 18.04 / zsh

Hardware / network configuration, and filesystems used.

ext 4

How much data is handled by borg?

Full borg commandline that lead to the problem (leave away excludes and passwords)

Describe the problem you're observing.

Impossibility to pass BORG_PASSCOMMAND using ohmyzsh and python virtual env

Can you reproduce the problem? If so, describe how. If not, describe troubleshooting steps you took before opening the issue.

1- Create a borg passphrase like indicated in the doc

echo "42spinAxis" >> ~/.borg-passphrase
chmod 400 ~/.borg-passphrase

2- Try to run borg create using PASSCOMMAND in a shell script :

export REPOSITORY=/home/xxx/folder/to/save
export BORG_PASSCOMMAND="$(cat ~/.borg-passphrase)"
borg create --compression zlib,9 ${REPOSITORY}::'{hostname}-{now:%Y-%m-%d}' /home/reyman/backup

Return this error :

passcommand supplied in BORG_PASSCOMMAND failed: [Errno 2] No such file or directory: '42spinAxis'

Ps : i also try without success :

export BORG_PASSCOMMAND="cat ~/.borg-passphrase"

[ThomasWaldmann]

There are multiple problems:

• if you use double quotes in your script, the shell will expand it immediately (which you may or may not want). if you do not want that, use single quotes.
• borg executes the given command directly as a subprocess, it does not give it to a shell.
• ~ might or might not refer to correct homedir, esp. if you use sudo.
• ~ is a shortcut expanded by the shell, so if you use that, make sure you have a shell and appropriate quoting so it is actually expanded by it.

[ThomasWaldmann]

unrelated, but maybe useful:

• use BORG_REPO instead of REPOSITORY, borg automatically uses that env var.
• zlib is a rather old algorithm, you maybe want to try zstd and its different levels.

[reyman]

There are multiple problems:

* if you use double quotes in your script, the shell will expand it immediately (which you may or may not want). if you do not want that, use single quotes.

* borg executes the given command directly as a subprocess, it does **not** give it to a shell.

* `~` might or might not refer to correct homedir, esp. if you use `sudo`.

* `~` is a shortcut expanded by the shell, so if you use that, make sure you have a shell and appropriate quoting so it is actually expanded by it.

Thx, i don't understand why this is the case in the faq which indicate only this :

export BORG_PASSCOMMAND="cat ~/.borg-passphrase"

Perhaps documentation could be modified with information you actually give me.

[ThomasWaldmann]

What happens if you try exactly like in FAQ?

[reyman]

What happens if you try exactly like in FAQ?

cat: '~/.borg-passphrase': Aucun fichier ou dossier de ce type
passcommand supplied in BORG_PASSCOMMAND failed: Command '['cat', '~/.borg-passphrase']' returned non-zero exit status 1

[ThomasWaldmann]

Hmm, that is unexpected. I am using bash, but encountering the same issue.

Although double quotes are used, ~ is not immediately expanded:

$ export BORG_PASSCOMMAND="cat ~/.borg-passphrase"
$ echo $BORG_PASSCOMMAND 
cat ~/.borg-passphrase

It works when using $HOME though, that gets immediately expanded:

$ export BORG_PASSCOMMAND="cat $HOME/.borg-passphrase"
$ echo $BORG_PASSCOMMAND 
cat /home/user/.borg-passphrase

cat (as usual on UNIX) is not expanding stuff internally, that is the task of the shell.

So, that is a documentation bug - do you want to make a pull request fixing it?

[ThomasWaldmann]

https://borgbackup.readthedocs.io/en/stable/usage/general.html#environment-variables

we could also add there that the command is executed directly, without using a shell.

[ThomasWaldmann]

https://www.gnu.org/software/bash/manual/bashref.html#Shell-Expansions

[PennRobotics]

I can't add too much insight, but these commands work in bash but not zsh, with the above error from @reyman :

      $( $BORG_PASSCOMMAND )
      `$BORG_PASSCOMMAND`

These commands work in both:

      eval $BORG_PASSCOMMAND
      $=BORG_PASSCOMMAND

This seems independent of file permissions, as 644 works identically to 400, and this is with $HOME instead of ~ in the env var. I even tried "which cat" and set $BORG_PASSCOMMAND to the full absolute path. This is a Mac w/ zsh and 1.1.8.

Current workaround is to use bash or BORG_PASSPHRASE.

[reyman]

Perhaps we could imagine some modifications on the borg source code to integrate command from any common shell, as found by @usfbrian ?

[PennRobotics]

For developers (I'm using a binary):
In key.py, is there a difference in how subprocess.check_output(shlex.split(passcommand), ...) is being called between zsh and bash? Either arguments or outputs?

Would this become the same for each shell by toggling the "posix" argument when split is called?

Something about how zsh handles spaces is different than bash.

[PennRobotics]

Ok... I don't know what changes would be necessary to implement this, but...
setopt sh_word_split
makes everything work with the existing syntax and everything.

Perhaps a check for $ZSH_VERSION, and then key.py runs different if true, or the tutorial includes a snippet about setopt, or zsh users just accept that life is hard.

That's all I've got!

[ThomasWaldmann]

borg's (python) code just takes whatever it gets in the BORG_PASSCOMMAND environment variable, splits it into command and args and executes that (it does not execute a shell implicitly).

So I don't see any change being necessary in the borg code.

[reyman]

Ok you're right :)