Flutterfire: [cloud_firestore] DocumentReference in startAfterDocument snapshot causes crash (invalid encode argument)

[creativecreatorormaybenot]

@amrfarid140 @ditman I think I have found another one of the .valueEncode issues as discussed here. See also: #1986.

The error I get is Invalid argument: Instance of 'DocumentReference'.

Full stack trace

E/flutter (14093): #0      StandardMessageCodec.writeValue (package:flutter/src/services/message_codecs.dart:392:7)
E/flutter (14093): #1      FirestoreMessageCodec.writeValue (package:cloud_firestore_platform_interface/src/method_channel/utils/firestore_message_codec.dart:83:13)
E/flutter (14093): #2      StandardMessageCodec.writeValue.<anonymous closure> (package:flutter/src/services/message_codecs.dart:389:9)
E/flutter (14093): #3      _LinkedHashMapMixin.forEach (dart:collection-patch/compact_hash.dart:379:8)
E/flutter (14093): #4      StandardMessageCodec.writeValue (package:flutter/src/services/message_codecs.dart:387:13)
E/flutter (14093): #5      FirestoreMessageCodec.writeValue (package:cloud_firestore_platform_interface/src/method_channel/utils/firestore_message_codec.dart:83:13)
E/flutter (14093): #6      StandardMessageCodec.writeValue.<anonymous closure> (package:flutter/src/services/message_codecs.dart:389:9)
E/flutter (14093): #7      _LinkedHashMapMixin.forEach (dart:collection-patch/compact_hash.dart:379:8)
E/flutter (14093): #8      StandardMessageCodec.writeValue (package:flutter/src/services/message_codecs.dart:387:13)
E/flutter (14093): #9      FirestoreMessageCodec.writeValue (package:cloud_firestore_platform_interface/src/method_channel/utils/firestore_message_codec.dart:83:13)
E/flutter (14093): #10     StandardMessageCodec.writeValue.<anonymous closure> (package:flutter/src/services/message_codecs.dart:389:9)
E/flutter (14093): #11     _LinkedHashMapMixin.forEach (dart:collection-patch/compact_hash.dart:379:8)
E/flutter (14093): #12     MapView.forEach (dart:collection/maps.dart:337:10)
E/flutter (14093): #13     StandardMessageCodec.writeValue (package:flutter/src/services/message_codecs.dart:387:13)
E/flutter (14093): #14     FirestoreMessageCodec.writeValue (package:cloud_firestore_platform_interface/src/method_channel/utils/firestore_message_codec.dart:83:13)
E/flutter (14093): #15     StandardMessageCodec.writeValue.<anonymous closure> (package:flutter/src/services/message_codecs.dart:389:9)
E/flutter (14093): #16     _LinkedHashMapMixin.forEach (dart:collection-patch/compact_hash.dart:379:8)
E/flutter (14093): #17     StandardMessageCodec.writeValue (package:flutter/src/services/message_codecs.dart:387:13)
E/flutter (14093): #18     FirestoreMessageCodec.writeValue (package:cloud_firestore_platform_interface/src/method_channel/utils/firestore_message_codec.dart:83:13)
E/flutter (14093): #19     StandardMethodCodec.encodeMethodCall (package:flutter/src/services/message_codecs.dart:524:18)
E/flutter (14093): #20     MethodChannel._invokeMethod (package:flutter/src/services/platform_channel.dart:148:13)
E/flutter (14093): #21     MethodChannel.invokeMethod (package:flutter/src/services/platform_channel.dart:329:12)
E/flutter (14093): #22     MethodChannel.invokeMapMethod (package:flutter/src/services/platform_channel.dart:356:48)
E/flutter (14093): #23     MethodChannelQuery.getDocuments (package:cloud_firestore_platform_interface/src/method_channel/method_channel_query.dart:87:46)
E/flutter (14093): #24     Query.getDocuments (package:cloud_firestore/src/query.dart:37:34)

Setup

The error happens every time a document has a DocumentReference as a value for a field and the snapshot is passed to startAfterDocument with this line (the getDocuments call causes it):

query.startAfterDocument(documentSnapshot).limit(8).getDocuments()

This was recently introduced with 0.13.0. I am currently using cloud_firestore version 0.13.3.

[ditman]

Hey @creativecreatorormaybenot, can you share what your query and documentSnapshot look like?

[creativecreatorormaybenot]

@ditman I have this debug log, which should be the exact same query (apart from the limit, but that value varies). Let me know if I should provide it in a different format.

Regarding the snapshot, I guess the most useful input here would be sharing the data types. The documents that are queried consist of:

Field types

• int
• float
• list (of strings)
• map (string: int)
• timestamp
• document reference

Some occur multiple times, e.g. multiple int fields.

This is for sure due to startAfterDocument and the document reference as a field of the document snapshot I pass. I am probably responsible for most of the code for this, but the refactoring seems to have broken it.

[ditman]

Yes, this is probably related to the document passed to startAfterDocument containing a DocumentReference. I'm surprised you say this only happens sometimes and not always, though!

[creativecreatorormaybenot]

@ditman Oh, I know why. Sorry for not thinking carefully.

The reason why it happens only sometimes is that the documents in the query mostly have the document reference field as null. So the error only occurs when a document reference is actually in Cloud Firestore. When the field is null, it does not happen (obviously).

---

Every time a document has a DocumentReference as a value for a field and the snapshot is passed to startAfterDocument, the error occurs.

[ditman]

Ah, cool, that makes sense! I'll take a look ASAP, if @amrfarid140 doesn't get to this before I do!

[ditman]

The issue seems to originate in _PlatformUtils::toPlatformDocumentSnapshot.

It's using the data getter from DocumentSnapshot, which internally converts Platform to Plugin types. This is for convenience for end-users (who only care about Plugin types), but it's a problem when we try to funnel that back into the plugin for further processing.

The fix is to either pass documentSnapshot._delegate.data (skips the getter), or to _CodecUtility.replaceValueWithDelegatesInMap(documentSnapshot.data).

I'm leaning towards the latter, which may waste a few cycles (and on big documents it might be a bunch) converting back and forth the data of the document snapshot, but it doesn't access private fields (which makes it harder to refactor the library as import/export based later).

[ditman]

The "fix" seems to break tests in web, so maybe it's not a good fix though:

Nope. The Firestore pagination integ test is broken in web in master already _(phew)_:

00:05 +7: Firestore pagination [E]
Invalid argument (dartObject): Could not convert: Instance of 'Timestamp'
package:dart-sdk/lib/_internal/js_dev_runtime/private/ddc_runtime/errors.dart 196:49      throw_
  package:firebase/src/utils.dart 119:3                                                     jsify
  package:dart-sdk/lib/internal/iterable.dart 417:29                                        elementAt
  package:dart-sdk/lib/internal/iterable.dart 221:19                                        toList
  package:firebase/src/firestore.dart 552:22                                                [_wrapPaginatingFunctionCall]
  package:firebase/src/firestore.dart 523:11                                                startAt
  package:cloud_firestore_web/src/query_web.dart 180:17                                     startAtDocument
  package:cloud_firestore/src/query.dart 130:17                                             startAtDocument
  package:firestore_example/main.dart 192:48                                                <fn>

[ditman]

My new test requires an index to be created. I'm waiting to be added to the Integ test project so I can create said index and continue on this.

[creativecreatorormaybenot]

@ditman How come that this was not caught in CI?
I realize that this is a Timestamp object and thus not completely related.

I could write tests to catch the error described in this issue if that helps.

[rbluethl]

I'm facing the exact same issue. Downgrading to cloud_firestore: ^0.12.11 works for now.

[ditman]

I'm going to push a fix for most of the issues I've found while investigating this (and fixing tests) today. I'll investigate the Timestamp issue next.

[ditman]

Tagged and published:

• https://pub.dev/packages/cloud_firestore/versions/0.13.4+1
• https://pub.dev/packages/cloud_firestore_web/versions/0.1.1+1

DocumentReferences with DocumentReferences in startAfterDocument should work as expected now! Please clear your pub caches and grab the latest versions, and let us know how it goes!

[creativecreatorormaybenot]

@ditman Works 🙂 Thank you so much :+1:

[ditman]

Sweet, thanks for reporting @creativecreatorormaybenot!

[kushvatsa]

not working, timestamp issue - error-Invalid argument (dartObject): Could not convert: Instance of 'Timestamp'

[ditman]

@kushvatsa can you cut a separate issue for that? Thanks!

[kushvatsa]

@ditman , #2153