Grpc: Error when deserializing message in Python (works with Go client)

[juanjux]

Should this be an issue in the gRPC issue tracker?

I think so.

What version of gRPC and what language are you using?

Python, 1.10.0 (tried with older versions with the same result).

What operating system (Linux, Windows, …) and version?

Linux Ubuntu 17.10, also reported happening on 16.04.

What runtime / compiler are you using (e.g. python version or version of gcc)

Python 3.6, binary whl packages installed with pip.

What did you do?

Trying to deserialize a certain binary message (named bad_msg.proto in the attached zip) produces this traceback:

ERROR:root:Exception deserializing message!
Traceback (most recent call last):
  File "/home/juanjux/pyenv/versions/3.6.0/lib/python3.6/site-packages/grpc/_common.py", line 87, in _transform
    return transformer(message)
google.protobuf.message.DecodeError: Error parsing message
Traceback (most recent call last):
  File "bblfshtest.py", line 5, in <module>
    pprint(client.parse("crash.java", language="java"))
  File "/home/juanjux/pyenv/versions/3.6.0/lib/python3.6/site-packages/bblfsh/client.py", line 74, in parse
    return self._stub.Parse(request, timeout=timeout)
  File "/home/juanjux/pyenv/versions/3.6.0/lib/python3.6/site-packages/grpc/_channel.py", line 487, in __call__
    return _end_unary_response_blocking(state, call, False, deadline)
  File "/home/juanjux/pyenv/versions/3.6.0/lib/python3.6/site-packages/grpc/_channel.py", line 437, in _end_unary_response_blocking
    raise _Rendezvous(state, None, None, deadline)
grpc._channel._Rendezvous: <_Rendezvous of RPC that terminated with (StatusCode.INTERNAL, Exception deserializing response!)>

The exact same message works perfectly with our Go client using the gRPC version for that language. I also tried the "bad" file with some online and local decoders and it didn't fail to deserialize.

Making a slight and trivial change on the input file in our pipeline so a slightly different binary message file is produced makes it work with the same code.

I've debugged the crash until gprc._common.py/transform(). There the received transformed argument is FromString from one of the native modules.

What did you expect to see?

Correct deserializing.

What did you see instead?

The exception above, with deserializing error and a StatusCode.Internal status.

Anything else we should know about your project / environment?

I've attached the nonworking binary message (bad_msg.proto') and a working one after a slight change (good_msg.proto`) to this issue:

good_bad_messages.zip

The schema of the binary message is ParseResponse from here:

https://github.com/bblfsh/sdk/blob/master/protocol/generated.proto#L48

The referenced Node object:

https://github.com/bblfsh/sdk/blob/master/uast/generated.proto#L11

[mehrdada]

This sounds like there's a possibility of this being a protobuf library issue (apparently, a new version of protobuf Python has been released in March). Which versions of protobuf library are you using on the client and server? Can you try reproducing the problem with just protobuf without gRPC? (Try serializing the message to a file in Go and reading it back with Python from the file, for example?)

[juanjux]

Before opening the issue I tried with several older python protobuf versions, but I'll try serializing from go and reading it back with Python and report.

[juanjux]

I exported from Go and tried to import from Python. It fails the same (error parsing message), so I'm closing this one and moving the issue to the protobuf repository, thanks for the help!