Google-cloud-node: Node.js gcloud/datastore/query returns RST_STREAM when .select and .filter are combined

[maximehk]

Doing a Projection Query with .select() and .filter() fails with error: "Received RST_STREAM err=2"

Reproduction steps:

• git clone https://github.com/GoogleCloudPlatform/nodejs-getting-started.git
• update the model-datstore.js file as follow

@@ -92,6 +92,7 @@ function list (limit, token, cb) {
   var q = ds.createQuery([kind])
     .limit(limit)
     .order('title')
+    .select(['title', 'description'])
     .start(token);

[stephenplusplus]

I believe this could be due to a couple of different issues with gRPC, the request library we use.

@eddavisson / @murgatroid99 -- does this sound like the newline bug you mentioned here? Or is this https://github.com/grpc/grpc/issues/4427 or something else?

[murgatroid99]

The error codes are listed in grpc.status. Assuming that that error code is from grpc, it stands for UNKNOWN, which just means that the server ended the stream with an error, but didn't provide a grpc status code. Unless there was a status message with the error, that's all it tells us.

[stephenplusplus]

Okay. @pcostell -- this would normally return the Precondition Failed error, wouldn't it?

[pcostell]

That's correct, this should return a Precondition Failed. But even with the existing bug you should get the error, just no error message.

In this case, the error message would have looked something like:

You did not have the required index for this query. Build the required index by adding the following to your index.yaml:

- kind: <kind>
   properties:
   - name: title
   - name: description

You need the composite index in this case because all the projected properties need to be in the index to be scanned. This is how we can return the results without requiring a lookup of the entire entity and is what allows this query to be very cheap.

[stephenplusplus]

@maximehacker that's your answer, so feel free to unsubscribe if the rest of this conversation gets boring :)

This is the error as it comes back to us from [email protected]:

{ Error: Received RST_STREAM err=2
    at /Users/stephen/dev/play/gissue-1533/nodejs-getting-started/2-structured-data/node_modules/grpc/src/node/src/client.js:417:17 code: 13, metadata: Metadata { _internal_repr: {} } }

Tested with grpc@1:

{ Error: {"created":"@1472231213.996835000","description":"RST_STREAM","file":"../src/core/ext/transport/chttp2/transport/frame_rst_stream.c","file_line":107,"http2_error":2}
    at /Users/stephen/dev/play/gissue-1533/nodejs-getting-started/2-structured-data/node_modules/gcloud/node_modules/grpc/src/node/src/client.js:417:17 code: 13, metadata: Metadata { _internal_repr: {} } }

[stephenplusplus]

@murgatroid99 @pcostell please let me know if I am missing something so I can get better errors to our users.

[Johanndutoit]

+1 for better errors here, was debugging and kinda assumed it was a missing index but a error actually telling me would go a long way. Thanks !

[stephenplusplus]

@murgatroid99 @pcostell I'm still stuck on how to pull out better errors from the API responses I'm getting (see two posts up). Any ideas?

[eddavisson]

I'm pretty sure this is the gRPC newline issue. It may be a while until that gets properly fixed in gRPC, but I am working on a workaround. I will post to https://github.com/GoogleCloudPlatform/google-cloud-datastore/issues/132 once that's in place.

[stephenplusplus]

Okay, thanks! I'll subscribe to that issue.

[1N50MN14]

@stephenplusplus I was just about to open exactly the same issue, ended up iterating through the results in all of my queries..

I'm also experiencing queries (really simple get-all-entries) randomly returning empty results when they shouldn't, is this related? this one has been bit of a PITA can't find any workarounds :( <-- EDIT: Never mind this one it was my fault (related to usage of keys)

[stephenplusplus]

@maximehacker can you see if the errors being returned are any better now? I believe a change was made upstream.

[stephenplusplus]

I've tested this and the expected error message is now being returned.