Borg: If Parent directory is missing the borg init fails on hetzner storagebox

[c33s]

Have you checked borgbackup docs, FAQ, and open Github issues?

yes

i believe i read a matching issue title but i can't find it again.

Is this a BUG / ISSUE report or a QUESTION?

kind of both

borg issue or hetzner issue?

System information. For client/server mode post info for both machines.

server is the hetzner backup space https://wiki.hetzner.de/index.php/BorgBackup/en according hetzner they use 1.1.6

Your borg version (borg -V).

`1.1.8

Operating system (distribution) and version.

debian stretch 9

Hardware / network configuration, and filesystems used.

ssh access, don't know the filesystem on hetzners side

How much data is handled by borg?

about 1mb

Full borg commandline that lead to the problem (leave away excludes and passwords)

root@myserver:~# borg init --encryption=keyfile-blake2 ssh://[email protected]/./myserver.your-storagebox.de/main

.ssh/config:

Host u12345.your-storagebox.de
    Port 23
    IdentityFile ~/.ssh/id-backupserver

Describe the problem you're observing.

if the parent directory does not exist the command fails. as soon as i create the parent directory myserver.your-storagebox.de manually everything works fine.

Can you reproduce the problem? If so, describe how. If not, describe troubleshooting steps you took before opening the issue.

yes, it happens all the time

Include any warning/errors/backtraces from the system logs

root@myserver:~# borg init --encryption=keyfile-blake2 ssh://[email protected]/./myserver.your-storagebox.de/main
Traceback (most recent call last):

  File "/.3LcwXLX0gNhr7CrW/python-envs/borg-1.1.6/lib/python3.6/site-packages/borg/remote.py", line 248, in serve
    res = f(**args)

  File "/.3LcwXLX0gNhr7CrW/python-envs/borg-1.1.6/lib/python3.6/site-packages/borg/remote.py", line 374, in open
    self.repository.__enter__()  # clean exit handled by serve() method

  File "/.3LcwXLX0gNhr7CrW/python-envs/borg-1.1.6/lib/python3.6/site-packages/borg/repository.py", line 182, in __enter__
    self.create(self.path)

  File "/.3LcwXLX0gNhr7CrW/python-envs/borg-1.1.6/lib/python3.6/site-packages/borg/repository.py", line 253, in create
    os.mkdir(path)

FileNotFoundError: [Errno 2] No such file or directory: '/home/myserver.your-storagebox.de/main'


Platform: Linux myserver 4.9.0-4-amd64 #1 SMP Debian 4.9.51-1 (2017-09-28) x86_64
Linux: debian 9.2
Borg: 1.1.8  Python: CPython 3.5.3
PID: 31710  CWD: /root
sys.argv: ['/usr/bin/borg', 'init', '--encryption=keyfile-blake2', 'ssh://[email protected]/./myserver.your-storagebox.de/main']
SSH_ORIGINAL_COMMAND: None

[ThomasWaldmann]

well, this is slightly out of scope of borg (which, like most tools, expect that the place where they create stuff actually exists).

otoh, it would be no big problem to add some commandline switch --create-parent-dirs to borg init if some services without direct ssh shell access do not offer some simple way to do this.

i would not make this the default behaviour though because the parent directory missing could also be an indication of failure outside of borg (e.g. if you back up to a mounted usb disk, the mount could just have failed, thus the parent directory not being there and you would not like to do the backup if that is the case).

[c33s]

as i am trying to create the parent directories with puppet on a hetzner storage box and easy stuff like ssh host mkdir -p simply doesn't work. hetzners answer was to use ftp to create the directory. -.-
with puppet i would also need to test if the directory exists to not raise an error on double creation.

so the --create-parent-dirs switch would really help very much. sounds good that it is not the default behavior but having this feature helps in such situations like i am in.

[ThomasWaldmann]

so, if the option is given, it would try to create all needed parent dirs (not the last path segment though as this is the repo directory itself and handled by subsequent code).

https://docs.python.org/3.4/library/os.html#os.makedirs

see borg.archiver.do_init.

[step21]

I implemented this. Sending a PR in a bit.

[ThomasWaldmann]

@step21 are you still working on it?

[ThomasWaldmann]

As there was no response, I took it.

I have to admit it was a bit less easy than I thought.

[ThomasWaldmann]

@step21 if you like, review #4235, compare it to whatever you have.