Dgraph: exploration mechanism to replace expand(__all__)

[Kingloko]

Experience Report

Note: Feature requests are judged based on user experience and modeled on Go Experience Reports. These reports should focus on the problems: they should not focus on and need not propose solutions.

What you wanted to do

Without having a deep understanding of the underlying schema I'd like to explore the data.

What you actually did

In the past you could do that by using expand(_all_).
Now, that only works if the node has a given type.

Why that wasn't great, with examples

If the node already has a type, the need to explore is mitigated. Because... well.. I can check the schema real quick. It is exactly when a node does not have a type that the need for exploration arises.
Granted, I can start using types, but without forcing types this seems like a problem that needs a solution.

Any external references to support your case

[Kingloko]

Since I feel like an old man complaining about new features... here's some humor?

[campoy]

Thanks for the issue, we're actually working on something that would fix your concerns without necessarily bringing back __predicate__ or expand(_all_) without types.

This would actually be brought into Ratel, allowing you to explore datasets easily without having to compromise the correctness of Dgraph.

Does that sound like a good solution?

[erhlee-bird]

I think I can understand the expand(_all_) being undesirable without explicit type information, but I don't see why __predicate__ would compromise the correctness of Dgraph.

I would strongly prefer having a programmatic/query-able method of performing a __predicate__ over having that functionality only supported within Ratel.

[Kingloko]

Thanks for the issue, we're actually working on something that would fix your concerns without necessarily bringing back __predicate__ or expand(_all_) without types.

This would actually be brought into Ratel, allowing you to explore datasets easily without having to compromise the correctness of Dgraph.

Does that sound like a good solution?

@campoy I have to echo @erhlee-bird's comment.
For the majority of cases I could type my nodes and be fine, but one of the features in my dataset is that some nods are dynamic by nature so my scripts use expand(_all_) to grab all the info that could be attached to those nodes.

I suspect I could get around this, by creating a "dump_all" type that has all the predicates. Thou, I'm not sure what the implication for that would be.

[martinmr]

I think I can understand the expand(_all_) being undesirable without explicit type information, but I don't see why __predicate__ would compromise the correctness of Dgraph.

I would strongly prefer having a programmatic/query-able method of performing a __predicate__ over having that functionality only supported within Ratel.

There are a few reasons why we got rid of _predicate_:

1. It was expensive to maintain. It involved keeping a list of all the predicates for each node in the graph. Depending on the structure of the graph, this can add a lot of extra data to dgraph.
2. Updates to this predicate were not guaranteed to be correct (as all other updates to Dgraph) because the choice was made to not send them through the usual raft consensus used for other updates due to performance reasons.

@Kingloko I am not sure what your application or data looks like but you could try adding a generic type (let's call it Node) that contains all the known predicates. Then add a predicate with this type for each node in the graph. Then expand(_all_) would try every known predicate. It might be expensive to maintain (probably no more expensive than the original _predicate_) but I think it could work.

[Kingloko]

@martinmr that's exactly what I was getting at when I said:
"I suspect I could get around this, by creating a "dump_all" type that has all the predicates. Thou, I'm not sure what the implication for that would be."

Thanks for getting back to me so quickly! Its reassuring that we both came up with approximately the same workaround.

[martinmr]

Sorry I missed that part. But if that solution works for you it's worth exploring and seeing if you get the expected results.

Unfortunately I don't think we can add something like that in Dgraph for the reasons stated in my last post. @campoy should we keep this issue open?

[martinmr]

Closing this issue since we have a possible workaround and the other feature @campoy mentioned is in ratel.

[campoy]

Let's keep it open for now until we have a fix, regardless of whether it ends up being on Ratel, other clients, or somewhere else.

[minhaj-shakeel]

Github issues have been deprecated.
This issue has been moved to discuss. You can follow the conversation there and also subscribe to updates by changing your notification preferences.