Flutterfire: [cloud_firestore] Always read from cloud not from cach

[niypoo]

I noticed My reads statics in firebase is very big Maybe in one day I read 2K documents , and I just one tested app have 17 documents in ios so I tracking the code and found that in android the firestore read only changed document from cloud but IOS it's always read from cloud so when I update one document the stream read 17 documents from cloud .

My Code

userListsStream(uid){
    Stream<QuerySnapshot> shoppingListsStream = 
    Firestore.instance.collection('lists').where('owner', arrayContains: uid).snapshots();

    shoppingListsStream.listen(
        (QuerySnapshot listsQuery) async {

          List<DocumentSnapshot> listsDocs = listsQuery.documents;
          if (listsDocs.length != 0) {
            //? foreach on lists Documents
            listsDocs.forEach(
              (DocumentSnapshot listDoc) async {
              print(listDoc.metadata.isFromCache ? "listDoc NOT FROM 
              NETWORK" : "listDoc FROM NETWORK");
              listItemsStream(listDoc.documentID);
         }
        )
       }
    })
}

listItemsStream(lid){
shoppingItemsRef =  Firestore.instance.collection('lists').document(lid).collection('items');

shoppingItemsRef.snapshots().listen(
        (QuerySnapshot itemsQuery) async {
          List<DocumentSnapshot> itemsDocs = itemsQuery.documents;
          if (itemsDocs.length != 0) {
            itemsDocs.forEach(
              (DocumentSnapshot itemDoc) async {
              print(itemDoc.metadata.isFromCache ? "itemDoc NOT FROM 
              NETWORK" : "itemDoc FROM NETWORK");
             }
           )
          }
}

and this is result after update one document

flutter: itemDoc FROM NETWORK
flutter: itemDoc FROM NETWORK
flutter: itemDoc FROM NETWORK
flutter: itemDoc FROM NETWORK
flutter: itemDoc FROM NETWORK
flutter: itemDoc FROM NETWORK
flutter: itemDoc FROM NETWORK
flutter: itemDoc FROM NETWORK
flutter: itemDoc FROM NETWORK
flutter: itemDoc FROM NETWORK
flutter: itemDoc FROM NETWORK
flutter: itemDoc FROM NETWORK
flutter: itemDoc FROM NETWORK
flutter: itemDoc FROM NETWORK
flutter: itemDoc FROM NETWORK
flutter: itemDoc FROM NETWORK
flutter: itemDoc FROM NETWORK
flutter: itemDoc FROM NETWORK

[ditman]

We're currently doing some big updates to cloud_firestore. Is this something that used to work earlier for you and started failing recently? Could you please let us know what version of the plugin are you using? (cc @amrfarid140)

[niypoo]

I have noticed this issue while using version 0.12.10+2 but I updated to last versions cloud_firestore: ^0.13.1 also this issue still exist only on IOS.
I have tested it on simulator and physical device IOS 12&13

[amrfarid140]

@niypoo just to check, Have you enabled persistence? .. you can do that by using Firestore#settings method.

Also, what happens on air plane mode. Do you still get data back?

[niypoo]

@amrfarid140
as default I didn't set persistence option but I set to to true as you told me

Firestore.instance.settings(
      persistenceEnabled: true,
    );

same result all from network when update one document.

but when turn on air plane mode it's read from cache and show that "NOT FROM NETWORK" and when return air plan off any update return read all from cloud

[ditman]

I have noticed this issue while using version 10.12.10+2

@niypoo do you mean 0.12.10+2? If you were seeing this before 0.13.1, this is probably a bug that is not related to @amrfarid140's change, and needs to be triaged normally.

[ditman]

/cc @collinjackson

[niypoo]

I have noticed this issue while using version 10.12.10+2

@niypoo do you mean 0.12.10+2? If you were seeing this _before_ 0.13.1, this is probably a bug that is not related to @amrfarid140's change, and needs to be triaged normally.

yes sorry i have updated it to "0.12.10+2" , yes I have this issue with "0.12.10+2" and still have after to update last version,

[Ramesh-X]

I'm using firestore version 0.13.3 and I also got the same error..

query.listen((docs) {
  if ((docs?.length ?? 0) > 0)
      print("From Cache: ${docs[0].metadata.isFromCache}");
});

When my phone is connected to the internet, this always print false.
But when the phone is offline, it will print true.

As explained in this video, Firestore listeners should always give the cached data first and try for data from server after that. But it is not happening here..

This is an essential feature in all our apps. We currently do manual caching or run all the query listeners at the startup in background.

[niypoo]

unfortunately , yesterday only I test my app with 83 documents read count over 8K ! only one user so 100 users I will broken .

[ditman]

@niypoo, I've read this SO answer, and it seems very relevant to this issue.

TL;DR:

By default, get() attempts to provide up-to-date data when possible by waiting for data from the server, but it may return cached data or fail if you are offline and the server cannot be reached.
This behavior can be altered via the Source parameter.

The source parameter is available both for Queries and DocumentReferences in the cloud_firestore plugin.

Have you tried retrieving your DocumentReference/Query.getDocument with Source.cache first?

According to the documentation on Source.cache, it seems that if you want a cache-first implementation, you'll have to manage it yourself:

Causes Firestore to immediately return a value from the cache, ignoring the server completely (implying that the returned value may be stale with respect to the value on the server). If there is no data in the cache to satisfy the [get()] or [getDocuments()] call, [DocumentReference.get()] will return an error and [Query.getDocuments()] will return an empty [QuerySnapshotPlatform] with no documents.

(Something like: try get from cache, else get from the default origin)

@Ramesh-X, thanks for the video, it explains the above as well!

Around minute 1:20, they say:

[...] if you're looking for a solution that is offline-first, or you're expecting your user to stay offline for months or years at a time, and then occasionally go online and backup their data, Cloud Firestore isn't probably the right solution for you.

Then in 3:37 they explain how to use the Source to "always fetch the cache data".

[ditman]

I'm closing this as "Firebase is working as designed". Feel free to reopen if the Source parameter doesn't work as advertised, though, it might be a bug in this plugin, or even in Firebase!

[Ramesh-X]

@ditman I'm not taking about getDocuments(). But snapshots() function.

If you see my code example I'm using Stream listen to listen to db changes.

In the video I provided, it is said that, if you use a listener, it will always get the cache data first before trying reach the online data (I can provide you the exact time in the video if you need). But if you run my example, you will see that it is not happening.

Hope you understood what I said. I think this issue should be re-opened !!

[ditman]

@Ramesh-X from what I've understood from the video you linked, and the official Firebase documentation, Firestore only fetches from cache first when the device is offline (you verified this earlier in this thread).

It seems the Firestore cache is not like a browser cache (cache first, then fetch from server). If you want a behavior like the browser cache, you need to do it yourself at the app level.

Am I understanding right that you want to get the cached data first even when the device is online when fetching snapshots? If so, this sounds like a new Firestore feature, and not a bug in the Flutter plugin.

[Ramesh-X]

@ditman Yeah, I got it..

If that feature haven't suggested by the Firestore community yet, it might not be as important as I thought it was. Anyway I have few workarounds that I have implemented in our mobile apps.

Thanks again for the reply..

[ditman]

Thanks for your help, and the report @Ramesh-X!

[RaviKavaiya]

@Ramesh-X I am facing the same issue. Can you tell me which workaround you implemented?

[RaviKavaiya]

@ditman Moreover, how should the documents be read from firestore in order to minimize read count. If I have 100 documents and call snapshots() 20 times, it will be counted as 2000 reads, even if the data is EXACTLY SAME as before as is already cached...

[ditman]

@RaviKavaiya it heavily depends on how cacheable the data that your app uses is, and how you've architected your app to allow for a caching layer to be added. I think this is still true (I'm not a firebase/firestore expert by any means):

It seems the Firestore cache is not like a browser cache (cache first, then fetch from server). If you want a behavior like the browser cache, you need to do it yourself at the app level.

Here's one example about caching locally with Firebase I found Googling a little bit (it talks about RTDB, but the principles should be the same for Firestore).

Also, please do not try to revive old issues; do open a new one instead, so more people gets to see it!