Borg: Corrupted repo causes borg check, delete fail

[verygreen]

I have this corrupted repo created with broked Fedora23 package on ppc64, where it segfaulted shortly after start.

Now both borg delete and borg check are really unhappy about this repo even after I did borg break-lock on it and removed the now stale lock.

I am attaching a sample. which is a tar inside of zip since github apparently does not support tar.gz files.
borg-repo.tar.zip

The errors I get are:

[root@powermac ~]# borg delete localhost:/tmp/backup
root@localhost's password: 
Remote: Borg 1.0.2: exception in RPC call:
Remote: Traceback (most recent call last):
Remote:   File "/usr/lib64/python3.4/site-packages/borg/remote.py", line 96, in serve
Remote:     res = f(*args)
Remote:   File "/usr/lib64/python3.4/site-packages/borg/repository.py", line 447, in get
Remote:     raise self.ObjectNotFound(id_, self.path) from None
Remote: borg.repository.ObjectNotFound: (b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', '/tmp/backup')
Remote: Platform: Linux powermac.localnet 4.4.8-300.fc23.ppc64 #1 SMP Wed Apr 27 17:14:46 UTC 2016 ppc64 ppc64
Remote: Linux: Fedora 23 Twenty Three
Remote: Borg: 1.0.2  Python: CPython 3.4.3
Remote: PID: 22394  CWD: /root
Remote: sys.argv: ['/usr/bin/borg', 'serve', '--umask=077']
Remote: SSH_ORIGINAL_COMMAND: None
Remote: 
Object with key 82 not found in repository localhost:/tmp/backup.
Traceback (most recent call last):
  File "/usr/lib64/python3.4/site-packages/borg/archiver.py", line 1498, in main
    exit_code = archiver.run(args)
  File "/usr/lib64/python3.4/site-packages/borg/archiver.py", line 1452, in run
    return args.func(args)
  File "/usr/lib64/python3.4/site-packages/borg/archiver.py", line 74, in wrapper
    kwargs['manifest'], kwargs['key'] = Manifest.load(repository)
  File "/usr/lib64/python3.4/site-packages/borg/helpers.py", line 93, in load
    cdata = repository.get(cls.MANIFEST_ID)
  File "/usr/lib64/python3.4/site-packages/borg/remote.py", line 354, in get
    for resp in self.get_many([id_]):
  File "/usr/lib64/python3.4/site-packages/borg/remote.py", line 358, in get_many
    for resp in self.call_many('get', [(id_,) for id_ in ids], is_preloaded=is_preloaded):
  File "/usr/lib64/python3.4/site-packages/borg/remote.py", line 263, in call_many
    raise Repository.ObjectNotFound(res[0], self.location.orig)
borg.repository.ObjectNotFound: (82, 'localhost:/tmp/backup')

Platform: Linux powermac.localnet 4.4.8-300.fc23.ppc64 #1 SMP Wed Apr 27 17:14:46 UTC 2016 ppc64 ppc64
Linux: Fedora 23 Twenty Three
Borg: 1.0.2  Python: CPython 3.4.3
PID: 22374  CWD: /root
sys.argv: ['/usr/bin/borg', 'delete', 'localhost:/tmp/backup']
SSH_ORIGINAL_COMMAND: None

[root@powermac ~]# borg check localhost:/tmp/backup
root@localhost's password: 
Local Exception.                
Traceback (most recent call last):
  File "/usr/lib64/python3.4/site-packages/borg/archiver.py", line 1498, in main
    exit_code = archiver.run(args)
  File "/usr/lib64/python3.4/site-packages/borg/archiver.py", line 1452, in run
    return args.func(args)
  File "/usr/lib64/python3.4/site-packages/borg/archiver.py", line 80, in wrapper
    return method(self, args, repository=repository, **kwargs)
  File "/usr/lib64/python3.4/site-packages/borg/archiver.py", line 147, in do_check
    last=args.last, prefix=args.prefix, save_space=args.save_space):
  File "/usr/lib64/python3.4/site-packages/borg/archive.py", line 663, in check
    self.key = self.identify_key(repository)
  File "/usr/lib64/python3.4/site-packages/borg/archive.py", line 696, in identify_key
    cdata = repository.get(next(self.chunks.iteritems())[0])
  File "borg/hashindex.pyx", line 341, in borg.hashindex.ChunkKeyIterator.__next__ (borg/hashindex.c:5407)
StopIteration

Platform: Linux powermac.localnet 4.4.8-300.fc23.ppc64 #1 SMP Wed Apr 27 17:14:46 UTC 2016 ppc64 ppc64
Linux: Fedora 23 Twenty Three
Borg: 1.0.2  Python: CPython 3.4.3
PID: 22419  CWD: /root
sys.argv: ['/usr/bin/borg', 'check', 'localhost:/tmp/backup']
SSH_ORIGINAL_COMMAND: None

[enkore]

The repository is completely empty (no segments) that's where the "borg check" crash comes from. Other commands give me this error:

borg.repository.Repository.ObjectNotFound: (
b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00',
'/home/mabe/Projekte/_oss/borg/green1043')

This error you show is interesting: borg.repository.ObjectNotFound: (82, 'localhost:/tmp/backup')

[verygreen]

Right.
I imagine this is a pretty well known state, though, for a completely empty state and perhaps asserting like this is not the most ideal way of dealing with it?

[enkore]

Yes, a check should be added that aborts earlier and with a more clear error message when a repository is opened that has no commits.

[ThomasWaldmann]

I fixed this as good as possible:

• borg list: clear error msg when there is no manifest, no traceback.
• borg check: works for empty repo now
• borg delete repo: works for empty repo (w/o manifest) now
• but: borg check --repair can not repair such repos. it has no chunks, thus can't determine key, thus can't write new manifest. but as repo delete works now, this can be solved by delete and fresh init.

https://github.com/borgbackup/borg/pull/1137/commits