Pulsar: Pulsar Client has no support in Mac 10.14.5 (18F132) ?

[ChethanUK]

Describe the bug
Python Pulsar Client say import _pulsar ImportError Reason: image not found

To Reproduce
Steps to reproduce the behavior:
In Mac: pip install pulsar-client --upgrade

python consumer.py

Traceback (most recent call last):
  File "consumer.py", line 1, in <module>
    import pulsar
  File "/Users/chethan/anaconda3/lib/python3.7/site-packages/pulsar/__init__.py", line 99, in <module>
    import _pulsar
ImportError: dlopen(/Users/chethan/anaconda3/lib/python3.7/site-packages/_pulsar.cpython-37m-darwin.so, 2): Library not loaded: /usr/local/opt/python/Frameworks/Python.framework/Versions/3.7/Python
  Referenced from: /Users/chethan/anaconda3/lib/python3.7/site-packages/_pulsar.cpython-37m-darwin.so
  Reason: image not found

Desktop (please complete the following information):

• OS: Mac 10.14.5 (18F132)

Additional context
Tried in both normal env and Conda env: 3.7, 2.7

Whats the possible solution?

Also, In official docs: pip install pulsar-client==2.3.2
But, I got error while installing:
Collecting pulsar-client==2.3.2

  ERROR: Could not find a version that satisfies the requirement pulsar-client==2.3.2 (from versions: 1.18.dev1, 1.19.dev1, 1.19.dev2, 1.19.0, 1.20.0, 1.21.0, 1.22.0, 2.0.1, 2.1.0, 2.1.1, 2.2.0, 2.2.1, 2.3.0, 2.3.0.post1, 2.3.1)
ERROR: No matching distribution found for pulsar-client==2.3.2

Thanks!.

[sijie]

Ah I see. we didn't upload the python clients for mac for 2.3.2 release. Can you try 2.3.1 client to get around the problem for now?

@jerrypeng since you were releasing 2.3.2, can you please check why we didn't upload the python clients for mac for 2.3.2?

2.3.2: https://pypi.org/project/pulsar-client/2.3.2/#files
2.3.1: https://pypi.org/project/pulsar-client/2.3.1/#files

[Guy-Dipietro-Bose]

This issue also occurs with 2.4.0. I'm unable to use any version greater than 2.2.1.

With 2.3.0 I see when running a test that import pulsar:
E ImportError: dlopen(../lib/python2.7/site-packages/_pulsar.so, 2): Symbol not found: __ZN6icu_636LocaleC1ERKS0_
E Referenced from: ../lib/python2.7/site-packages/_pulsar.so
E Expected in: flat namespace

With 2.3.1 I see no error but pytest fails to run:
I see Empty test suite.

I don't see any errors, but this impacts pytest in a way that it fails to discover tests. This issue doesn't occur with

[amrfaisal]

Any updates on this?

[aahmed-se]

Is this still an issue ?
It's working fine for me.

[lxy1992]

We met the some problem.

• Mac OS 10.14.6
• python 3.7.4
• pulsar-client 2.4.1
  I can install it perfectly, but when I run the server locally, it shows me the following error

import _pulsar
ImportError: dlopen(/Users/xxx/.local/share/virtualenvs/hunger-game-N1jOgOYL/lib/python3.7/site-packages/_pulsar.cpython-37m-darwin.so, 2): Library not loaded: /usr/local/opt/protobuf/lib/libprotobuf-lite.20.dylib
  Referenced from: /Users/xxxx/.local/share/virtualenvs/hunger-game-N1jOgOYL/lib/python3.7/site-packages/_pulsar.cpython-37m-darwin.so
  Reason: image not found

[aahmed-se]

@lxy1992 try this

brew install protobuf

I will take a look at why the static linking didn't work correctly

[nathan-zhu]

same error for me in mac 10.14.6

import pulsar
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/xxx/miniconda/lib/python3.7/site-packages/pulsar/__init__.py", line 102, in <module>
    import _pulsar
ImportError: dlopen(/Users/xxx/miniconda/lib/python3.7/site-packages/_pulsar.cpython-37m-darwin.so, 2): Library not loaded: /usr/local/opt/protobuf/lib/libprotobuf-lite.20.dylib
  Referenced from: /Users/xxx/miniconda/lib/python3.7/site-packages/_pulsar.cpython-37m-darwin.so
  Reason: image not found

i already installed protobuf with brew.

[aahmed-se]

@nathan-zhu try this

pip3 install pulsar-client==2.4.1.post1

[thlautenschlaeger]

for me it's still not working

[thlautenschlaeger]

only works with python 3.6 and pulsar 2.0.1 but missing mandatory updates

[RAbraham]

@nathan-zhu try this

pip3 install pulsar-client==2.4.1.post1

works on python 3.7, Mojave

[cabello]

I am having the same issue, just following the documentation trying to product some messages.

• MacOS: 10.15.2
• Pulsar: 2.4.2 on Docker 19.03.5
• Python: 3.7.2
• protobuf: stable 3.11.2
• pulsar-client: 2.4.2

pulsar-env ❯ python consume.py
Traceback (most recent call last):
  File "consume.py", line 1, in <module>
    import pulsar
  File "/Users/me/.pyenv/versions/pulsar-env/lib/python3.7/site-packages/pulsar/__init__.py", line 102, in <module>
    import _pulsar
ImportError: dlopen(/Users/me/.pyenv/versions/pulsar-env/lib/python3.7/site-packages/_pulsar.cpython-37m-darwin.so, 2): Library not loaded: /usr/local/opt/[email protected]/lib/libprotobuf-lite.9.dylib
  Referenced from: /Users/me/.pyenv/versions/3.7.2/envs/pulsar-env/lib/python3.7/site-packages/_pulsar.cpython-37m-darwin.so
  Reason: image not found

And contents of the consume.py script:

import pulsar

client = pulsar.Client('pulsar://localhost:6650')
consumer = client.subscribe('my-topic',
                            subscription_name='my-sub')

while True:
    msg = consumer.receive()
    print("Received message: '%s'" % msg.data())
    consumer.acknowledge(msg)

client.close()

I tried changing the protocol from pulsar:// to http://, but the import call is failing so it doesn't reach that point.

Any suggestions?

[cabello]

I also tried compiling & installing the Python library from source after brew install libpulsar MacOS C++ libraries documentation and Python documentation but I got the following error:

pulsar-env ❯ python setup.py install
running install
running bdist_egg
running egg_info
writing pulsar_client.egg-info/PKG-INFO
writing dependency_links to pulsar_client.egg-info/dependency_links.txt
writing requirements to pulsar_client.egg-info/requires.txt
writing top-level names to pulsar_client.egg-info/top_level.txt
reading manifest file 'pulsar_client.egg-info/SOURCES.txt'
writing manifest file 'pulsar_client.egg-info/SOURCES.txt'
installing library code to build/bdist.macosx-10.14-x86_64/egg
running install_lib
running build_py
running build_ext
error: [Errno 2] No such file or directory: '_pulsar.so'

[zzzming]

Latest update with this line up working on MacOS 10.14.6

MacOS: 10.14.6
Python 2.7.16
Pip 19.3.1 from /Library/Python/2.7/site-packages/pip-19.3.1-py2.7.egg/pip (python 2.7)
Pulsar client: 2.4.1

=======
I had to manually install these software. If you are on vanilla OS, the newer version of protobuf, version 3.11 in my case, would be installed. This is critical to align up with python client's _pulsar.so. Python client 2.4.1 and 2.4.2 require protobuf version 2, (dot version .20 and .22 respectively). Protobuf 2 is no longer brew install-able.

Since 2.4.1.post1 can work with protobuf 3, these steps would work.

brew install libpulsar
sudo pip install -U --ignore-installed six
sudo pip install pulsar-client==2.4.1.post1

Producer and consumer with SSL enabled with our production were tested and I just manually copied the root CA certificate in PEM format to a local file as following.

trust_certs='./ca-certificates.crt'
client = pulsar.Client(service_url,
                        authentication=pulsar.AuthenticationToken(token),
                        tls_trust_certs_file_path=trust_certs)

It also works on python3 with this command the client and macOS and python version.
MacOS: 10.14.6
pip 19.3.1 from /usr/local/lib/python3.7/site-packages/pip (python 3.7)
Python 3.7.6

pip3 install pulsar-client==2.4.1.post1

I hope this helps.

[damienburke]

Any other work arounds for this? None of the above work for me (have not attempted to build from source). Thanks

[sijie]

We have already started supporting 10.14 since 2.5.x. I have verified that 2.5.2 works locally at my laptop. (10.14.6).

Can you try the latest release?