Orbit-db: Support write-permissioned databases

Created on 24 Jul 2016 · 19Comments · Source: orbitdb/orbit-db

The goal here is to add write-permissioned databases to OrbitDB. It'll mean that we can have per-user databases or define a set of keys that can write to a given database.

enhancement

Source

haadcode

Most helpful comment

This was finally released! 🎉

haadcode on 28 Nov 2017

🎉3

All 19 comments

https://github.com/haadcode/ipfs-log/issues/34

haadcode on 24 Jul 2016

Dropping this from uPort integration milestone as this is not _needed_ to reach the milestone. Definitely want to do this at some point though.

haadcode on 30 Aug 2016

I opened this issue for a similar need in luckychain code: https://github.com/ipfs/interface-ipfs-core/issues/120

mitar on 3 Apr 2017

👍1

Thanks @mitar, this is indeed something we need (on multiple levels).

I plan to work on this in the coming months. The idea is that the signing key will work as the "owner" of the database, ie. only updates signed with that key will be processed and accepted in orbit-db (made a quick & dirty prototype some time ago in ipfs-log for signed messages). This should give us functionality that allows permissioned logs and as such, permissioned databases.

If anyone wants to get this done sooner or otherwise work on it, ping me and I'd be more than happy to help you implement this in orbit-db/ipfs-log.

haadcode on 3 Apr 2017

@haadcode I've been having similar thoughts!

But rather that one could make use of a user's public key in constructing keys in the key-value store; somehow authenticating requests (e.g. signing the transaction rather than the entire database) to that particular key in the kv-store, but it's quite vague for me at the moment. Maybe that's essentially what you mean with the term permissioned databases.

Basically, what I want is for individual users to be able to own parts of a database — but not the entire thing.

(Typically, what I've done previously in e.g. Redis, is prefix a user key before the attribute so cblgh-attributes, cblgh-data etc; a similar idea could apply to the idea I'm talk about above.)

cblgh on 3 Apr 2017

I think ssb does this well. Each user has its own hashlog, and only signed messages by them can go into it. And then the whole database is a collection of such hashlogs. I think with this you will get what you are describing above, @cblgh.

mitar on 3 Apr 2017

Interesting - thanks @mitar! I'll have a closer look at ssb!

cblgh on 3 Apr 2017

I think ssb has some other issues (not having great IPFS underneath), so I think orbitdb is a better approach. But we can see good ideas in other projects as well. :-)

mitar on 3 Apr 2017

👍2

Yeah, what I have in mind is similar to ssb's approach, but with the distinction that signed logs don't form a "database from collections of such hashlogs" but rather that in orbit-db a database is a collection of database operations ("transactions") which in turn means the database is one log. That log can written to by anyone who has the specified key(s), eg. "/my-cool-twitter-app/haadcode/QmSignKey1" is a database only the key "QmSignKey1" can write to. We can have a mechanism to allow multiple keys to write to a log, eg. "/my-cool-twitter-app/haadcode/[QmSignKey1, QmSignKey2]" can be written to by owners of "QmSignKey1" and "QmSignKey2". In both, the name of the database is the "path", eg. "/my-cool-twitter-app/haadcode/QmSignKey1".

signing the transaction rather than the entire database

Yup, this is how it would happen. Each transaction (in orbit-db's case a database operation is a transaction) is signed, added to the operations log of that database (ipfs-log), then added to the database (upon which the signature is verified against the given key(s)) and the updated database information is sent to the pubsub topic. When peers listening for updates receive the update "new entry: QmFoo", they will verify the entry's signature against the publish signing key and if it checks, it'll be considered a valid entry that can be added to the oplog and to the database. Hope this clarifies it a bit.

Basically, what I want is for individual users to be able to own parts of a database — but not the entire thing.

I don't think you'll be able to use what I've been describing here per-key so that users(/keys) own a database partly. However, what you can do is to separate the databases on the ID level (database name) so that each user has their own database, eg. "haad-keys-and-values" and "cblgh-keys-and-values". Would that work for you? If not, let me know more about the use case of per-user keys in a kvstore and we can think about possible approaches. I haven't really thought it that way yet but I'm sure we can implement it if needed. I believe this would hapen purely on orbit-db level and not necessarily on the ipfs/data structures level.

As for key distribution, there's Keystore that has been planned for some time, and I think we got a first-implementation here but I haven't tested it. My idea was to leave the key distribution part to IPFS, and do all of that via keystore in IPFS.

Then, on the pubsub level, there will be a way to have "access-based" pubsub topics in a similar way that only certain nodes/keys are allowed to publish to a topic, which will give another layer of verification, but I believe from orbit-db's perspective this is more helpful to eg. prevent spam. I believe the design is not quite locked down yet on Topic Descriptors, so not sure how it'll all fit together with this but at least it'll help with the replication part.

Let me know if this clarifies it or if you have any further questions. Highly appreciate your comments and thoughts, so let's keep discussing in order to make sure the use cases are covered and we have a good idea for the approach :)

@cblgh @mitar do you JS? :) Do you think this would be something you'd want to try to tackle and implement? :) If so, let me know and I'd be happy to work with you on it. If not, no worries, I'll get to it myself sooner or later.

haadcode on 4 Apr 2017

Ah, sadly I do not have any time. But I have recently implemented a blockchain on top of IPFS myself, so this is why it got me looking into other similar projects, like orbit.

mitar on 4 Apr 2017

However, what you can do is to separate the databases on the ID level (database name) so that each user has their own database, eg. "haad-keys-and-values" and "cblgh-keys-and-values".

This could work if it's possible to differentiate & read data from different databases easily in orbit; what I would want to do is basically allow for people to own various subdomains (at least logically); bandcamp could be a good example, where each artist has access to their subdomain but not the entirety of bandcamp. If it's easy to work between many (scale being at least in the hundreds, preferably more since 1 db = 1 user) different orbit databases within the same app my use case would fit pretty good, if be somewhat of a hack (which I must admit I am occasionally fond of). I'd also love for it to happen at the orbit-level of things as that feels like that might simplify a lot.

do you JS? :) Do you think this would be something you'd want to try to tackle and implement? :) If so, let me know and I'd be happy to work with you on it. If not, no worries, I'll get to it myself sooner or later.

I do! I've been wanting to dig into orbit to! I'll have some free time around the end of the month, so I'd love to poke your mind and get some help with setting up the essentials so that I can start playing around and implement various things (I'm thinking low-hanging fruit the first few days until I feel comfortable enough with bigger features).

Let me know if you want me to clarify anything!

cblgh on 4 Apr 2017

@haadcode, what you are describing is then more like matrix? It seems they have a similar idea of common hashchain of signed messages, where you can temporary fork, but then new messages link to any of previous forks, and create an even growing DAG, merging together as one long hashchain with some occasional uncles.

mitar on 6 Apr 2017

A question about the replication. Let say each user of an app need their own signed space to publish updates. If I understand correctly, that would mean one orbit database for each with a restricted access based on their private key.

Who would replicate this database ? Everyone that use orbit-db ? Or is it opt-in for each user to select the database they care about ?

Ideally for my use case, that would be the same database for all users of this app, but each user have a private namespace available. Data replication would be enough to maintain data online for a few days without everyone having a full copy of the database. Is that in the scope of orbit-db ?