Firebase-android-sdk: Firestore [21.5.0]: Long fetch times with persistence enabled

[AurimasSikorskas]

[REQUIRED] Step 2: Describe your environment

• Android Studio version: 4.0.1
• Firebase Component: Firestore
• Component version: 21.5.0

[REQUIRED] Step 3: Describe the problem

After upgrading firestore dependency 21.4.3 -> 21.5.0
Firestore document fetch times became waaaayy longer.
ex: in my project, we have documents with a 3 element array, and depending on which element was changed, the fetch time increases
array 0 element was changed, document is fetched in 0.8sec
array 1 element was changed, document is fetched in 20 sec
array 2 element was changed, document is fetched in 35 sec

At least this is the behavior I see on my main projects development environment, on the production environment it only takes 5~7 seconds, not sure why...

This document is only changed by the back end.
If persistence is disabled, all fetch times become almost instant

Steps to reproduce:

Made a mock project to reproduce a similar issue. the database is updated from the client, but we can still see long delays.
Note: if I switch dependency to 21.4.3 or turn persistence off everything works super smooth

Scenario 1:

1. Run mock app
2. Refresh list (for me load time is 11 sec)
3. Add 'volume'
4. Refresh list (for me load time is 16 sec)
5. Add 'volume'
6. Refresh list (for me load time is 22 sec)
7. Add 'volume'
8. Refresh list (for me load time is 30 sec)
9. update 'book'
10. Refresh list (for me load time is 30 sec)

each new volume increases load time by ~5.5 sec
doing a data fetch without changing anything in the database, return the data instantly.

Scenario 2:

if any of the volume data is updated by using firestore console, depending on the volume position(array index) fetch time increases.
example:
volume 0 : 0 sec
volume2 : 1 sec
volume4 : 2 sec
volume8 : 4 sec

[thebrianchen]

@AurimasSikorskas Thanks for taking the time to create a repro! I was able to reproduce this locally and observe the behavior you described. I'm going to dig into this some more, and report back once I understand what's happening.

[thebrianchen]

Quick follow up: the slow load times are due to our recent upgrade to the protolite library in #1534. We are currently using the Protobuf equality checker, which is the root cause of the delay. When I tested your repro app with a custom build that used Firestore's equality checker, the update speeds returned to their 21.4.3 times. I'll work on getting a fix out in the coming week, which should hopefully allow you to upgrade without such a drastic performance regression.

Also, thanks so much for taking the time to provide such an easy-to-use repro app and description! I found it very helpful when debugging the issue and trying to find a fix.