Protobuf: TypeError: __new__() got an unexpected keyword argument 'serialized_options'

[davenso]

After compiling proto for Python target, I get the following error when running either client or server code. What could be the problem? (OS is CentOS 7.4).

protoc --python_out=. --grpc_out=. --plugin=protoc-gen-grpc=/usr/local/bin/grpc_python_plugin engine.proto

import engine_pb2
 File "/home/dell/viotWare/rpc/grpc/engine_pb2.py", line 22, in <module>
   serialized_pb=_b('\n\x16engine.proto\x12\tinfengine\"\x1c\n\x0cInferRequest\x12\x0c\n\x04\x64\x61ta\x18\x01 \x01(\x0c\"\x1d\n\x0bInferResult\x12\x0e\n\x06result\x18\x01 \x01(\t2M\n\x0fEngine\x12:\n\x05Infer\x12\x17.infengine.InferRequest\x1a\x16.infengine.InferResult\"\x00\x42\x39\n\x1finfengineB\x0eInfEngineProtoP\x01\xa2\x02\x03IENb\x06proto3')
TypeError: __new__() got an unexpected keyword argument 'serialized_options'

protoc --version
libprotoc 3.6.0

python --version
Python 3.6.5

[acozzette]

@davenso I suspect this is a symptom of using generated code that is too new for the runtime--i.e. the protoc version used is newer than the library (older versions of the library don't know about that serialized_options parameter). With Python the library version has to be at least as new as the protoc version.

[anandolee]

This error happens when new generated code is being used with an old runtime, That's a configuration we don't support. Upgrade protobuf runtime to 3.6.0 or above can solve the error.

For compatibility, we support older generated code with newer protobuf runtime.

[aliYaHf]

@anandolee File "/home/anaconda3/lib/python3.6/site-packages/tensorboardX/proto/resource_handle_pb2.py", line 22, in <module> serialized_pb=_b('\n(tensorboardX/proto/resource_handle.proto\x12\x0ctensorboardX\"r\n\x13ResourceHandleProto\x12\x0e\n\x06\x64\x65vice\x18\x01 \x01(\t\x12\x11\n\tcontainer\x18\x02 \x01(\t\x12\x0c\n\x04name\x18\x03 \x01(\t\x12\x11\n\thash_code\x18\x04 \x01(\x04\x12\x17\n\x0fmaybe_type_name\x18\x05 \x01(\tB/\n\x18org.tensorflow.frameworkB\x0eResourceHandleP\x01\xf8\x01\x01\x62\x06proto3') TypeError: __init__() got an unexpected keyword argument 'serialized_options'
Ubantu16.04
whereis protoc
protoc: /home/anaconda3/bin/protoc
protoc --version
libprotoc 3.4.0
protobuf-3.4.0
Should I update protobuf or other problems with me?

[Atlowell]

I am encountering this issue today using Python 3.5.2

I have been using the precompiled version of protoc obtained from your release page and protobuf runtime obtained through pip3 (though I also tried downloading and building it myself with the same results). They are both running version 3.9.2 (confirmed using protoc --version and pip3 show protobuf. I also tried downgrading both to version 3.6.0 with the same results.

I obtain the issue when running a setuptools utility for an external project that works fine without the inclusion of protobuf. I have confirmed that setuptools is obtaining and building the correct version of the protobuf library.

Using /usr/lib/python3/dist-packages Searching for protobuf==3.9.2 Best match: protobuf 3.9.2 Adding protobuf 3.9.2 to easy-install.pth file

The error message:

Traceback (most recent call last): File "/usr/local/bin/TRACE-tp-python", line 11, in <module> load_entry_point('sawtooth-TRACE==1.2.1.dev1', 'console_scripts', 'TRACE-tp-python')() File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 542, in load_entry_point return get_distribution(dist).load_entry_point(group, name) File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2569, in load_entry_point return ep.load() File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2229, in load return self.resolve() File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2235, in resolve module = __import__(self.module_name, fromlist=['__name__'], level=0) File "<frozen importlib._bootstrap>", line 969, in _find_and_load File "<frozen importlib._bootstrap>", line 958, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 664, in _load_unlocked File "<frozen importlib._bootstrap>", line 634, in _load_backward_compatible File "/usr/local/lib/python3.5/dist-packages/sawtooth_TRACE-1.2.1.dev1-py3.5.egg/sawtooth_TRACE/processor/main.py", line 37, in <module> File "<frozen importlib._bootstrap>", line 969, in _find_and_load File "<frozen importlib._bootstrap>", line 958, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 664, in _load_unlocked File "<frozen importlib._bootstrap>", line 634, in _load_backward_compatible File "/usr/local/lib/python3.5/dist-packages/sawtooth_TRACE-1.2.1.dev1-py3.5.egg/sawtooth_TRACE/processor/transaction/handler.py", line 32, in <module> File "<frozen importlib._bootstrap>", line 969, in _find_and_load File "<frozen importlib._bootstrap>", line 958, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 664, in _load_unlocked File "<frozen importlib._bootstrap>", line 634, in _load_backward_compatible File "/usr/local/lib/python3.5/dist-packages/sawtooth_TRACE-1.2.1.dev1-py3.5.egg/sawtooth_TRACE/processor/state.py", line 30, in <module> File "<frozen importlib._bootstrap>", line 969, in _find_and_load File "<frozen importlib._bootstrap>", line 958, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 664, in _load_unlocked File "<frozen importlib._bootstrap>", line 634, in _load_backward_compatible File "/usr/local/lib/python3.5/dist-packages/sawtooth_TRACE-1.2.1.dev1-py3.5.egg/sawtooth_TRACE/processor/protobuf/adminaccount_pb2.py", line 23, in <module> TypeError: __new__() got an unexpected keyword argument 'serialized_options'

UPDATE:

The issue was caused by two versions of protobuf runtime being used: the one that I installed with setuptools and one imported from another project during setup. The imported version was the one being used. Downgrading protoc to 3.4.0 resolved the issue.

If anyone experiences this in the future, make sure you're not inadvertently importing an older version.