Dgraph: Support logical replication / change data capture (CDC) with Kafka
The following workflow is becoming more and more common in streaming architectures:
If Dgraph is to be used as a primary data store in a production system, it must support logical replication / change data capture.
Is this currently possible? If not, is it planned for the future?
fritzblue
All 5 comments
It's not possible right now but we can do something like this in the future. Some users have asked for subscriptions. This would be a similar feature I suppose.
pawanrawal
on 19 Feb 2018
We could potentially support this at Badger level first, and then expose it via Dgraph.
manishrjain
on 15 Jan 2019
After chatting with @gitlw, it seems like this could be implemented relatively easily on top on our current plans for datacenter to datacenter replication, which already uses Kafka.
Adding it to our 1.X plans!
campoy
on 11 Jul 2019
For an example of a distributed DB that's doing this in what I would argue is the Right Wayâ„¢, see CockroachDB https://www.cockroachlabs.com/docs/stable/change-data-capture.html
fritzblue
on 23 Oct 2019
Github issues have been deprecated.
This issue has been moved to discuss. You can follow the conversation there and also subscribe to updates by changing your notification preferences.
minhaj-shakeel
on 20 Jul 2020
Related issues
mbudge
·
3Comments
jimanvlad
·
5Comments
jerodsanto
·
3Comments
marvin-hansen
·
4Comments
pjebs
·
4Comments
Most helpful comment
After chatting with @gitlw, it seems like this could be implemented relatively easily on top on our current plans for datacenter to datacenter replication, which already uses Kafka.
Adding it to our 1.X plans!