Borg: How to gracefully kill borg on remote machine

[marcpope]

Have you checked borgbackup docs, FAQ, and open Github issues?

Yes

Is this a BUG / ISSUE report or a QUESTION?

Question

System information. For client/server mode post info for both machines.

Backing up from a client machine to a server over network

Your borg version (borg -V).

1.1.3 (I know I need to upgrade).

Operating system (distribution) and version.

CentOS 7.6

Hardware / network configuration, and filesystems used.

Local Network, GB Connection, CentOS 6.7, default LVM type filesystems

How much data is handled by borg?

500-600gb in this instance, about 6.7million files, doesn't seem to matter in this case.

Full borg commandline that lead to the problem (leave away excludes and passwords)

borg create --list someuser@hostname:reponame::e9gKdQDJ-tjU7ZKwvebfq2mLWNkOy /chroot/home
(note: private info redacted)

Describe the problem you're observing.

I don't know if this is necessarily a problem, or a feature. Let's say a backup is in progress. The client is backing up to the server and the client crashes or restarts. The server leaves the process open, for a period of time. (Eventually after a few hours, roughly, it seems to timeout). Would there be a safe way to kill this process and free the locks? Should anything be done to make sure the next backup is clean or can another backup just continue?

Can you reproduce the problem? If so, describe how. If not, describe troubleshooting steps you took before opening the issue.

Yes, everytime.

Include any warning/errors/backtraces from the system logs

n/a

[ThomasWaldmann]

you can maybe solve this on ssh level.
if the borg client dies, this means the ssh client is dying also.
if the ssh server checks the connection regularly, it should notice the client and the connection has died and terminate itself also, also terminating the server side "borg serve" process.

see #636 which mentions some of these ssh settings.

[marcpope]

so if the server can see that the client is no longer running the borg process, is it safe to kill the process? then run borg break-locks on the repo? I have found that the server eventually terminates at some predefined period on it's own, but it seems to be quite a while (maybe an hour or more).

[ThomasWaldmann]

If there is no ongoing activity / no borg client connected, yes it should be safe to kill the process.

Preferably, send it a SIGTERM.

There is some code in borg that should clean up locks when it terminates, so you maybe do not even have to remove them afterwards.

[mikenye]

See issue #3988.

The SSH client & server settings documented in my latest reply to that issue should prevent the borg serve process from sticking around after the client disconnects.

[ThomasWaldmann]

Somebody could check if we have docs or a faq entry about this and if not, do a PR to add some.

[mikenye]

I'd love to do this, however this would be the first time I've created a pull request... :-)
I'll do some research into how to do this unless you've got time to give me some instructions.

[ThomasWaldmann]

Guess you can find the generic info on the web.

For borg, PRs usually are made against master branch and might be backported to 1.1-maint branch later.

[mikenye]

Done, first ever pull request in. Be kind. :)

[mikenye]

The documentation has been updated, I think this issue can now be closed?

[marcpope]

Thank you. I think this will work for me.