Pulsar: Python Pulsar client appears to throw regular `Exception`s instead of `PulsarException`s

[grantwwu]

Expected behavior

The Python client should throw a subclass of Exception, not just Exception.

Actual behavior

An Exception is thrown. See, for example:

In [2]: p = pulsar.Client()

In [3]: c = p.get_consumer("foo")
2018-12-04 17:24:48.860 INFO  ConnectionPool:63 | Created connection for pulsar://localhost:6650
2018-12-04 17:24:48.871 ERROR ClientConnection:329 | [<none> -> pulsar://localhost:6650] Failed to establish connection: Connection refused
2018-12-04 17:24:48.872 INFO  ClientConnection:1237 | [<none> -> pulsar://localhost:6650] Connection closed
2018-12-04 17:24:48.872 ERROR ClientImpl:374 | Error Checking/Getting Partition Metadata while Subscribing- 5
2018-12-04 17:24:48.872 INFO  ClientConnection:195 | [<none> -> pulsar://localhost:6650] Destroyed connection

< deleted internal wrapper code >

Exception: Pulsar error: ConnectError

To demonstrate it's not a subclass:

In [8]: try:
   ...:     c.receive(timeout_millis=500)
   ...: except Exception as e:
   ...:     print(type(e))
   ...:
<class 'Exception'>

Steps to reproduce

Easiest way is probably supply a bogus URL to get a ConnectError

System configuration

Pulsar version: 2.2.0 for both Pulsar and the Python client

[legendtkl]

I would like to work on this issue

[grantwwu]

Ah, well, I'm not going to stop you :) I tried to look at the source, but couldn't figure out how the Python source interacted with the C++ library off the top of my head, or where the bare Exceptions were coming from.

[srkukarni]

As @grantwwu said, Pulsar's python client is a simple wrapper around c++.
You can find the c++ code in https://github.com/apache/pulsar/tree/master/pulsar-client-cpp. The actual implementations are in https://github.com/apache/pulsar/tree/master/pulsar-client-cpp/lib. For example https://github.com/apache/pulsar/blob/master/pulsar-client-cpp/lib/Message.cc.
The python interactions are specified by https://github.com/apache/pulsar/blob/master/pulsar-client-cpp/python/src/message.cc and the actual python code is at https://github.com/apache/pulsar/blob/master/pulsar-client-cpp/python/pulsar/__init__.py#L138
Hope that helps

[legendtkl]

@grantwwu I just read the source code, and find that pulsar use boost.python to translate C++ to python，including the Exception.
Maybe you can follow the files:

• pulsar-client-cpp/lib/client.cc //C++ source implement
• pulsar-client-cpp/python/pulsar/__init__.py // python code
• pulsar-client-cpp/python/src/client.cc // translate
• pulsar-client-cpp/python/src/pulsar.cc // translate
• pulsar-client-cpp/python/src/utils.h // translate

all the c++ exception, include

case ResultOk:
            return "Ok";

        case ResultUnknownError:
            return "UnknownError";

        case ResultInvalidConfiguration:
            return "InvalidConfiguration";

        case ResultTimeout:
            return "TimeOut";

        case ResultLookupError:
            return "LookupError";

        case ResultConnectError:
            return "ConnectError";

        case ResultAuthenticationError:
            return "AuthenticationError";

        case ResultAuthorizationError:
            return "AuthorizationError";

        case ResultErrorGettingAuthenticationData:
            return "ErrorGettingAuthenticationData";

        case ResultBrokerMetadataError:
            return "BrokerMetadataError";

        case ResultBrokerPersistenceError:
            return "BrokerPersistenceError";

        case ResultConsumerBusy:
            return "ConsumerBusy";

        case ResultNotConnected:
            return "NotConnected";

will be tranlate into PyExc_Exception

[legendtkl]

And I wonder if any work should be do for the issue ? @srkukarni @jiazhai

[merlimat]

@legendtkl ideally the exception should be wrapped in a PulsarException, so that application can catch a specific exception instead of a blanket catch. Also, for example when getting a timeout, one should be able to check the type of the exception

[legendtkl]

@merlimat yeah, got it. A custom Exception PulsarException might be ok for this. I will try for that.

[xgdgsc]

I' m trying tls connection and get Pulsar error: ConnectError which isn' t very informative of what config could be wrong. Digging server logs for now.