Google-cloud-node: gRPC max size limit

[pyhrus]

Since grpc version 1.0.0, message size is limit 4MB, and pubsub itself supports payload size upto 10MB. If there is message that is greater than 4MB in queue, pubsub client errors out with Error: Received message larger than max (6374241 vs. 4194304)

Environment details

• OS: Debian Jessie
• Node.js version: 6.3.1
• npm version:3.10.3
• google-cloud-node version: 0.8.0

Steps to reproduce

1. Publish message greater than 4MB using HTTP API
2. Use pubsub client to pull the message

Thanks!

[stephenplusplus]

@murgatroid99 @jmuk do you know if there's a way to handle this?

[jmuk]

It's really obscure to me how to customize the parameters like this. Maybe the third parameter to the Client constructor might be used (https://github.com/grpc/grpc/blob/master/src/node/src/client.js#L732) but I don't know what values or structures can configure this.

@murgatroid99, do you know which keys/values we should use here?

[murgatroid99]

The keys and values are defined at https://github.com/grpc/grpc/blob/master/include/grpc/impl/codegen/grpc_types.h#L148. The one you want for this case is "grpc.max_send_message_length".

[jmuk]

Thanks!

I also filed https://github.com/googleapis/gax-nodejs/issues/107 for this problem.

[stephenplusplus]

PR sent to remove the limit of the max message sent length and max message received length: #2007

[arbesfeld]

We still seem to be hitting this 4mb limit while testing with the following versions:

    "@google-cloud/bigtable": "0.10.1",
    "@google-cloud/pubsub": "0.11.0",
    "@google-cloud/storage": "1.1.0"

Any recommendation for how to avoid this GRPC limit?

[Michael-Jing]

I am using the grpcio python lib, and encounter the same issue, any idea to get rid of this?