Borg: create specs for backup service?

[ThomasWaldmann]

On IRC, the idea of a (non-professional, for free, mutual) backup service based on borg came up (note that more professional backup services could use some of this, too).

If some people have internet connection, some free disk space and borg+ssh set up, they could use the other's space for their backups, while offering their space for the others' backups.

We could have this as better alternative to "individually reinvent":

• standard, documented, reviewed setups (keeping data and networks as safe as possible, making setup straightforward, enable easy migration to new backup partners if needed)
• standard "forms" to negotiate such agreements (so that the mutual expectations / offerings are somehow in sync)

Setups

• secure borg+ssh configuration
• borg versions support
• fs layout (at least the "remotely visible" part that affects configuration)

• fs quota setup?

  Forms

• storage space wanted / offered

• connectivity wanted / offered (pub IP v4 / v6 / static / dyndns / up/down bandwidth)
• software wanted / offered (e.g. Linux, *BSD, ..., borg version / setup)
• hardware wanted / offered (e.g. raspi vs. server, single disk vs. RAID, ...)
• environment wanted / offered (e.g. at home, company, datacenter)
• mail-a-disk wanted / offered (for big volumes, initial seed and/or desaster case)

• country / location (might be interesting for net latency, desaster case, legal reasons)

  Misc ideas

• Maybe it is simpler if everybody just specifies what they have to offer?

• Use a separate github repo for this?
  
  
  
  • Repo might have documents, configs, ... (if we do not want to add them to borg docs).
  
  
  • Issue tracker might have offers (and requests).
  
  
• Mention using an external hard drive for easy mail-a-disk?

---

:moneybag: there is a bounty for this

[henfri]

So, I make a start.
I offer up to 5TB on a Linux Server with 50Mbit upstream, 100Mbit down stream.

[ThomasWaldmann]

as documenting this is quite some work, I'ld consider putting a small bounty on this IF multiple people are interested in this issue - in the hope that other people might increase the bounty.

so, to show your interest, add reaction "heart" for this comment.

update: I put up a bounty for this.

[s200999900]

I can share up to 2Tb on linux server with up/down stream ~50Mb/s.

[RonnyPfannschmidt]

for the standard setup, i would suggest to program/test them via tools like nix/docker/ansible instead of letting people set them up manually/custom

my personal preference would be to spec it as a nix module and ship that as docker/rkt/vm for others to consume

[ThomasWaldmann]

@RonnyPfannschmidt the implementation can be done however someone does it. this ticket is about the spec and it will have to be as a text document.

[RonnyPfannschmidt]

either the spec is executable/testable and convergent, or you are in hell ^^
at least thats my experience when you let people implement a spec

[ThomasWaldmann]

I don't think giving the spec as an implementation of it is a good idea. If you use nix, about 0.1% of potential users will be able/willing to use it. Similar for any other deployment tool / container / VM technology, but maybe with a bit higher percentage.

Thus, specifying in a text document how the service has to look from the outside is easier. And if someone implements incompatibly, it's not my problem. ;-)

Based on that, we can add specific implementations, but that's not in scope of this ticket.

[RonnyPfannschmidt]

hmm, i feel there is at least a need for some automated test for such implementations,
unless there is a automated test people must pass, its very certain a messy situation with lots of barely working implementations is going to happen

[enkore]

Can always have a spec (authoritative) and a reference test suite or implementation (or both)

But first there needs to be a working spec

[ThomasWaldmann]

Let's continue there: https://github.com/borgbackup/service-spec - there is an issue tracker also.

[ThomasWaldmann]

closing due to #5230.

[ThomasWaldmann]

reopening, see #5230.