Google-cloud-dotnet: ListDocuments with ShowMissing for delete nested collection

[Thaina]

Under the low level API, the ListDocumentRequest can specified ShowMissing argument to query deleted document that still contain subcollection. It needed for clearing whole database so please have high level API for it

[jskeet]

@schmidt-sebastian: Can you comment on this please?

[schmidt-sebastian]

This matches a more general feature request of the Query API that is already on our roadmap. We will eventually allow querying the entire subtree of documents, which will then include data hidden away in empty documents.

We can likely prioritize the specific use case (empty documents at the collection level) as it is indeed required to enumerate all Firestore data. I will update this issue after our internal discussion.

[Thaina]

@schmidt-sebastian Hello, how the discussion was going?

[schmidt-sebastian]

@Thaina Thanks for following up. We are still waiting for more use feedback in order to prioritize this feature.

[Thaina]

@schmidt-sebastian Currently now I have a problem that firebase console cannot delete all document from collection that contains documents all contain subcollection. It seem my collection is too large and when I command it to delete everything it just got spinning circle (more than 8 hours)

While it was a bug in console, if the ability to list missing document was available in firestore sdk from the start I could just workaround in the meantime. It now is my blocking issue even I have file this issue 3 months ago

This is actually a hidden bug for such times though. If I happen to create a document with subcollection. And delete it with sdk by query all document. Then I cannot have anyway at all to get the subcollection programmatically. Which is make an invisible cost to anyone that don't come in to look at console and use delete button manually (which also broken right now)

And then when I reuse its name, the subcollection also resurrected and make a subtle bug that I got the old data that was not related to the new data

Even in the beta period this bug should not happen. If it has too many problem and uncontrollable like this you should just hide its usage from public from the start. When firebase document mention it people like me then try to use it as structural tree in the sense that it would be easy to manage. But it proven that it completely wrong. It completely useless

We can likely prioritize the specific use case (empty documents at the collection level) as it is indeed required to enumerate all Firestore data. I will update this issue after our internal discussion.

This is what I was expect to know from march and even now it still has no answer? What internal discussion you have done?

[Thaina]

And while @jskeet was trying to stop me from using low level API. I then was constraint myself to not use it and try to solve everything as high level API. I put my trust in your work and changed all my code to the high level ones

But then nothing good come at me. Low level API let me be productive while the high level API just kept making my life harder

[Thaina]

Not to mention you have talking about

querying the entire subtree of documents,

for 2 months. And currently now it has no future. It never got mention. We actually know that there was a plan to do it for a year but it but no one ever have seen it progressed. And you don't even talk about it again here

[jskeet]

If you want to keep using the low level API, go ahead - but please understand I won't be able to support you in doing so.

Also, please be respectful of the work that both the Firestore team and the Google Cloud Client libraries team are doing. We have many, many competing demands on our time, and there are only so many hours in the day. I understand your frustration, but responses of this kind aren't productive in my experience.

[Thaina]

This is what happened in network after opened around 2 minutes

On Thu, May 17, 2018 at 2:25 PM, Jon Skeet [e-mail] wrote:

If you want to keep using the low level API, go ahead - but please
understand I won't be able to support you in doing so.

Also, please be respectful of the work that both the Firestore team and
the Google Cloud Client libraries team are doing. We have many, many
competing demands on our time, and there are only so many hours in the day.
I understand your frustration, but responses of this kind aren't productive
in my experience.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/GoogleCloudPlatform/google-cloud-dotnet/issues/1905#issuecomment-389770777,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AA_oS5NqTKfkWg8iECHH4nAImTlvwMkCks5tzSXqgaJpZM4SKEWF
.

[mkelle]

Was this ever resolved, I have a similar issue.. Is there a way to get the path of all empty docs in a collection? https://stackoverflow.com/questions/51121513/firestore-cant-get-empty-docs

[jskeet]

@mkelle: I have no new information. I believe the Firestore team is still evaluating this as an option.

(There's no point using @ in a Stack Overflow question when the intended users aren't already involved in the question, by the way - and GitHub usernames won't work there either.)

[Thaina]

@mkelle Not only we don't have proper API. We could only use REST API (or maybe grpc). And the REST API only provide you the ListDocuments with ShowMissing without the API to show _only missing_ documents. Which mean if you want to clear the missing docs, you need to query the whole collection and filter deleted document in the client side

[mkelle]

@jskeet: ok, I hope they will come out with a solution soon. And ok, I will remember that, thanks for letting me know.

@Thaina: my issue is actually the other way around, I want to access an use the empty docs, because they all have a subcollection which I need. But it seems like their is no way to list them (the document paths), or to list all the documents. When I query the collection, I can only get the result to include docs that contains data, not empty doc that only contain a subcollection.

[schmidt-sebastian]

FYI, this issue is still on our table and we plan to add a first-party implementation for this API somewhat shortly.

[jskeet]

Further FYI, I'm hoping to start on the implementation for this really soon.

[mkelle]

sounds good @jskeet 👍

[jskeet]

Now finally implemented :) What level of demand is there for a new release that includes this?

[jskeet]

Available in version 1.0.0-beta14.

[mkelle]

Nice jskeet, thank you!

[DominicTobias]

@jskeet Thanks for this, does it work with firestore JS API? how do I specify this option in the query?

// Where? Can get accept it? The docs suggest not. * edit: yep it errors
await projectSnap.ref.collection('cluster').get({ showMissing: true });

[jskeet]

@DominicTobias: I'm afraid I don't know anything about the JS API - I suggest you ask on the Firebase Slack channel or Google Group

[schmidt-sebastian]

The Web JS client do not support this feature. It is available via the Server SDK for Node: https://cloud.google.com/nodejs/docs/reference/firestore/1.1.x/CollectionReference#listDocuments