Borg: AssertionError & PermissionError

[hubitor]

Have you checked borgbackup docs, FAQ, and open Github issues?

Yes

Is this a BUG / ISSUE report or a QUESTION?

BUG

System information. For client/server mode post info for both machines.

Your borg version (borg -V).

borg 1.1.6

Operating system (distribution) and version.

Arch Linux

Hardware / network configuration, and filesystems used.

How much data is handled by borg?

350 GB

Full borg commandline that lead to the problem (leave away excludes and passwords)

borg list /mybackups/Borg/home/

Describe the problem you're observing.

I've installed borgmatic today and tried it for the first time. It has created a backup without error messages. After that I wanted to list my archives. I normally don't use sudo for this. But I got an AssertionError and PermissionError from Borg as a normal user. If use sudo I can list my archives without any errors. To me it makes sense that Borg complained with PermissionError since apparently borgmatic created the last archive as a root user but the AssertionError doesn't.

Can you reproduce the problem? If so, describe how. If not, describe troubleshooting steps you took before opening the issue.

Include any warning/errors/backtraces from the system logs

$ borg list /mybackups/Borg/home/ 
Exception ignored in: <bound method Repository.__del__ of <Repository /mybackups/Borg/home>>
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/borg/repository.py", line 174, in __del__
    assert False, "cleanup happened in Repository.__del__"
AssertionError: cleanup happened in Repository.__del__
Local Exception
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/borg/repository.py", line 1266, in get_fd
    return self.fds[segment]
  File "/usr/lib/python3.6/site-packages/borg/lrucache.py", line 21, in __getitem__
    value = self._cache[key]  # raise KeyError if not found
KeyError: 50873

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/borg/archiver.py", line 4312, in main
    exit_code = archiver.run(args)
  File "/usr/lib/python3.6/site-packages/borg/archiver.py", line 4244, in run
    return set_ec(func(args))
  File "/usr/lib/python3.6/site-packages/borg/archiver.py", line 137, in wrapper
    with repository:
  File "/usr/lib/python3.6/site-packages/borg/repository.py", line 184, in __enter__
    self.open(self.path, bool(self.exclusive), lock_wait=self.lock_wait, lock=self.do_lock)
  File "/usr/lib/python3.6/site-packages/borg/repository.py", line 406, in open
    if segment is not None and self.io.get_segment_magic(segment) == ATTIC_MAGIC:
  File "/usr/lib/python3.6/site-packages/borg/repository.py", line 1298, in get_segment_magic
    fd = self.get_fd(segment)
  File "/usr/lib/python3.6/site-packages/borg/repository.py", line 1268, in get_fd
    fd = open(self.segment_filename(segment), 'rb')
PermissionError: [Errno 13] Permission denied: '/mybackups/Borg/home/data/5/50873'

Platform: Linux hostess 4.16.13-2-ARCH #1 SMP PREEMPT Fri Jun 1 18:46:11 UTC 2018 x86_64
Linux: arch Arch Linux
Borg: 1.1.6  Python: CPython 3.6.5
PID: 2711  CWD: /home/hose
sys.argv: ['/usr/sbin/borg', 'list', '/mybackups/Borg/home/']
SSH_ORIGINAL_COMMAND: None

[ThomasWaldmann]

You likely have a permissions issue. Always use same user to access the repo.

On June 12, 2018 4:12:00 PM GMT+02:00, hubitor [e-mail] wrote:

Have you checked borgbackup docs, FAQ, and open Github issues?

Yes

Is this a BUG / ISSUE report or a QUESTION?

BUG

System information. For client/server mode post info for both

machines.

Your borg version (borg -V).

borg 1.1.6

Operating system (distribution) and version.

Arch Linux

Hardware / network configuration, and filesystems used.

How much data is handled by borg?

350 GB

Full borg commandline that lead to the problem (leave away

excludes and passwords)
borg list /mnt/BACKUPS/Borg/home/

Describe the problem you're observing.

I've installed borgmatic today and tried it for the first time. It has
created a backup without error messages. After that I wanted to list my
archives. I normally don't use sudo for this. But I got an
AssertionError and PermissionError from Borg as a normal user. If use
sudo I can list my archives without any errors. To me it makes sense
that Borg complained with PermissionError since apparently borgmatic
created the last archive as a root user but the AssertionError
doesn't.

Can you reproduce the problem? If so, describe how. If not,

describe troubleshooting steps you took before opening the issue.

Include any warning/errors/backtraces from the system logs

$ borg list /mybackups/Borg/home/
Exception ignored in: /mybackups/Borg/home>>
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/borg/repository.py", line 174,
in __del__
assert False, "cleanup happened in Repository.__del__"
AssertionError: cleanup happened in Repository.__del__
Local Exception
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/borg/repository.py", line 1266,
in get_fd
return self.fds[segment]
File "/usr/lib/python3.6/site-packages/borg/lrucache.py", line 21, in
__getitem__
value = self._cache[key] # raise KeyError if not found
KeyError: 50873

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/borg/archiver.py", line 4312, in
main
exit_code = archiver.run(args)
File "/usr/lib/python3.6/site-packages/borg/archiver.py", line 4244, in
run
return set_ec(func(args))
File "/usr/lib/python3.6/site-packages/borg/archiver.py", line 137, in
wrapper
with repository:
File "/usr/lib/python3.6/site-packages/borg/repository.py", line 184,
in __enter__
self.open(self.path, bool(self.exclusive), lock_wait=self.lock_wait,
lock=self.do_lock)
File "/usr/lib/python3.6/site-packages/borg/repository.py", line 406,
in open
if segment is not None and self.io.get_segment_magic(segment) ==
ATTIC_MAGIC:
File "/usr/lib/python3.6/site-packages/borg/repository.py", line 1298,
in get_segment_magic
fd = self.get_fd(segment)
File "/usr/lib/python3.6/site-packages/borg/repository.py", line 1268,
in get_fd
fd = open(self.segment_filename(segment), 'rb')
PermissionError: [Errno 13] Permission denied:
'/mybackups/Borg/home/data/5/50873'

Platform: Linux hostess 4.16.13-2-ARCH #1 SMP PREEMPT Fri Jun 1
18:46:11 UTC 2018 x86_64
Linux: arch Arch Linux
Borg: 1.1.6 Python: CPython 3.6.5
PID: 2711 CWD: /home/hose
sys.argv: ['/usr/sbin/borg', 'list', '/mybackups/Borg/home/']
SSH_ORIGINAL_COMMAND: None

--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/borgbackup/borg/issues/3894

--
Sent from my Android device with K-9 Mail. Please excuse my brevity.

[hubitor]

Well I wrote that before but this long error message doesn't seem right to me. I think borg should at least throw another message.

[ThomasWaldmann]

There are obviously a lot of places in borg repository code that access repo files.

If you create a mixup of permissions in the repo (some stuff root, some other user(s)), such a permission denied error can also happen earlier or later, depening on your permissions mix. so, there is no quick way to check if permissions are right as there are a lot of repo files.

Not sure if we should handle that by a very high level handler just to make it "pretty". Pretty error messages can have the problem to hide useful information (for developers).

About the AssertionError: this is because the repo object is disposed by __del__ which is not the usual way we finalize this. It is caused by the other exception here and you can ignore it.

[hubitor]

How can I check which user created the repo and which user created the archives? I can't figure out how the ownership of the repo has suddenly changed.

[ThomasWaldmann]

ls -lR repo

[hubitor]

This is what I get with ls -lR repo:

ls -lR /mybackups/Borg/home/                                                                       2 

/mybackups/Borg/home/ :
total 50520
-rw------- 1 hose hose      635 Aug 21  2017 config
drwx------ 8 hose hose     4096 Jun 10 22:49 data
-rw------- 1 root root   191327 Jun 12 22:17 hints.50957
-rw------- 1 root root 51516698 Jun 12 22:17 index.50957
-rw------- 1 root root      190 Jun 12 22:17 integrity.50957
-rw------- 1 root root       16 Jun 12 22:55 nonce
-rw------- 1 hose hose       26 Aug 21  2017 README

/mybackups/Borg/home/ data:
total 1168
drwx------ 2 hose hose 155648 Aug 21  2017 0
drwx------ 2 hose hose 253952 Aug 21  2017 1
drwx------ 2 hose hose 245760 Aug 21  2017 2
drwx------ 2 hose hose 253952 Jun 10 23:24 3
drwx------ 2 hose hose 253952 Jun 10 23:22 4
drwx------ 2 hose hose  16384 Jun 12 22:17 5

/mybackups/Borg/home/ data/0:
total 63823384
-rw------- 1 hose hose  5461375 Aug 21  2017 1
-rw------- 1 hose hose  5794328 Aug 21  2017 10
-rw------- 1 hose hose  5254470 Aug 21  2017 100
-rw------- 1 hose hose 11387833 Aug 21  2017 1000

...

/mybackups/Borg/home/ data/5:
total 2473912
-rw------- 1 hose hose       17 Jun 10 22:54 50316

...

-rw------- 1 hose hose  5242921 Jun 11 16:09 50757
-rw------- 1 hose hose       17 Jun 11 16:09 50758
-rw------- 1 hose hose       17 Jun 11 16:09 50760
-rw------- 1 hose hose       17 Jun 11 16:10 50762
-rw------- 1 root root       17 Jun 12 11:48 50764
-rw------- 1 root root       17 Jun 12 11:48 50766
-rw------- 1 root root       17 Jun 12 11:48 50768

• config, data and README are owned by user hose
• hints,index, integrity and nonce are owned by user root

And also from the backup files it`s obvious that the ownership has changed from user hose to user root on 12.06.2018.
The question now is what are my options since it has been mentioned multiple time that it is not good to have mixed repo ownership and not root ownership. Should I delete the whole repo and create a new one with a normal user or is it sufficient to delete the archives user root created?

[ThomasWaldmann]

• decide on which user (group) to use in future (and ALWAYS use that from now on)
• chown -R thatuser.thatgroup REPO
• borg check REPO

If the check succeeds then I guess there is no reason to start from scratch.

[hubitor]

I've done this:

chown -R thatuser.thatgroup REPO
borg check REPO

and there is no output for borg check. Does this mean is everything OK?
When I create a backup there are no errors but the exit code is 1. What does this mean?
If indeed something is wrong how can I get more debug info?

[ThomasWaldmann]

rc 1 means warning and there should be a warning in the output.

[hubitor]

Sorry I don't understand. borg check shouldn't return a message for error or not? In my case where there is no output what is it?

On the other hand borg create seems to be successful but exit code is 1. There is no warning.

[ThomasWaldmann]

as this is a separate issue and looks like unrelated to this issue, please open a new one and give the commands how to reproduce the issue, also borg version etc.