Pulsar: python bookkeeper client defective for python2 in mac

[srkukarni]

Describe the bug
Importing bookkeeper.admin module fails on a mac running python2
To Reproduce
Mac running Mojave and python2.
make sure to have python-client 2.3.0.post1 installed.
try doing
import bookkeeper.admin

it fails with
Traceback (most recent call last):
File "", line 1, in
File "/Users/sanjeevkulkarni/Library/Python/2.7/lib/python/site-packages/bookkeeper/admin/__init__.py", line 15, in
from bookkeeper.admin.client import Client
File "/Users/sanjeevkulkarni/Library/Python/2.7/lib/python/site-packages/bookkeeper/admin/client.py", line 20, in
from bookkeeper import types
File "/Users/sanjeevkulkarni/Library/Python/2.7/lib/python/site-packages/bookkeeper/types.py", line 22, in
from bookkeeper.proto import common_pb2
File "/Users/sanjeevkulkarni/Library/Python/2.7/lib/python/site-packages/bookkeeper/proto/common_pb2.py", line 22, in
serialized_pb=_b('\n\x0c\x63ommon.proto\x12\x17\x62ookkeeper.proto.common\"*\n\x08\x45ndpoint\x12\x10\n\x08hostname\x18\x01 \x01(\t\x12\x0c\n\x04port\x18\x02 \x01(\x05\x42-\n)org.apache.bookkeeper.stream.proto.commonP\x01\x62\x06proto3')
TypeError: __init__() got an unexpected keyword argument 'serialized_options'

Expected behavior
the import should work
Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• OS: [e.g. iOS]

Additional context
Add any other context about the problem here.

[jerrypeng]

@sijie @ivankelly @jiazhai can you please take look as this prevents running functions on MacOS using python2

[sijie]

@jerrypeng @srkukarni I am not able to reproduce what your problem at my mac using the 4.9.0 bookkeeper client. my mac is Mojave 10.14.

do you have a different grpc proto3 installed? the stack trace shows it is related to proto3. Also I am not sure what does 2.3.0.post1 mean here. It doesn't seem to be an official release.

[aahmed-se]

2.3.0.post1 is official release it's an update to 2.3.0

[jerrypeng]

@sijie @srkukarni after I updated protobuf to the latest version for python2, I didn't get the error anymore:

Jerrys-MacBook-Pro:incubator-pulsar jerrypeng$ pip install protobuf --upgrade
Collecting protobuf
  Using cached https://files.pythonhosted.org/packages/2b/2b/d51219eb18a140836cb656053e5408cd18fd752217ff73ca596204cd3183/protobuf-3.6.1-cp27-cp27m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl
Requirement already satisfied, skipping upgrade: setuptools in /usr/local/lib/python2.7/site-packages (from protobuf) (40.6.3)
Requirement already satisfied, skipping upgrade: six>=1.9 in /usr/local/lib/python2.7/site-packages (from protobuf) (1.11.0)
Installing collected packages: protobuf
  Found existing installation: protobuf 3.5.1
    Uninstalling protobuf-3.5.1:
      Successfully uninstalled protobuf-3.5.1
Successfully installed protobuf-3.6.1

Jerrys-MacBook-Pro:incubator-pulsar jerrypeng$ python2
Python 2.7.15 (default, Dec 27 2018, 11:56:32) 
[GCC 4.2.1 Compatible Apple LLVM 10.0.0 (clang-1000.11.45.5)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import bookkeeper.admin
>>> 

[sijie]

@jerrypeng maybe we can set the version of protobuf to be 3.6.1

[aahmed-se]

https://pypi.org/project/protobuf/

that's the latest version so it should fine to take a dependency on it.

[srkukarni]

That seems to be the right approach to make sure that protobuf is atleast 3.6.1