Sqlalchemy: Type error on PG8000 connection

[BlackLight]

Describe the bug

After upgrading to SQLAlchemy to 1.3.19 I'm encountering issues when connecting to PostgreSQL over the pg8000 driver - not sure whether the issue lies in sqlalchemy or pg8000 though. The connect() call itself fails over select version(). No connection issues occur on any of the previous versions of SQLAlchemy.

Expected behavior

Connection should be successful.

To Reproduce

>>> from sqlalchemy import create_engine
>>> db = create_engine('postgresql+pg8000://user:pwd@ip/dbname', echo=True)
>>> conn = db.connect()

Error

2020-10-13 18:06:33,790 INFO sqlalchemy.engine.base.Engine select version()
2020-10-13 18:06:33,790 INFO sqlalchemy.engine.base.Engine ()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 2265, in connect
    return self._connection_cls(self, **kwargs)
  File "/usr/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 104, in __init__
    else engine.raw_connection()
  File "/usr/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 2371, in raw_connection
    return self._wrap_pool_connect(
  File "/usr/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 2338, in _wrap_pool_connect
    return fn()
  File "/usr/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 304, in unique_connection
    return _ConnectionFairy._checkout(self)
  File "/usr/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 778, in _checkout
    fairy = _ConnectionRecord.checkout(pool)
  File "/usr/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 495, in checkout
    rec = pool._do_get()
  File "/usr/lib/python3.8/site-packages/sqlalchemy/pool/impl.py", line 140, in _do_get
    self._dec_overflow()
  File "/usr/lib/python3.8/site-packages/sqlalchemy/util/langhelpers.py", line 68, in __exit__
    compat.raise_(
  File "/usr/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
    raise exception
  File "/usr/lib/python3.8/site-packages/sqlalchemy/pool/impl.py", line 137, in _do_get
    return self._create_connection()
  File "/usr/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 309, in _create_connection
    return _ConnectionRecord(self)
  File "/usr/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 440, in __init__
    self.__connect(first_connect_check=True)
  File "/usr/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 664, in __connect
    pool.dispatch.first_connect.for_modify(
  File "/usr/lib/python3.8/site-packages/sqlalchemy/event/attr.py", line 314, in exec_once_unless_exception
    self._exec_once_impl(True, *args, **kw)
  File "/usr/lib/python3.8/site-packages/sqlalchemy/event/attr.py", line 285, in _exec_once_impl
    self(*args, **kw)
  File "/usr/lib/python3.8/site-packages/sqlalchemy/event/attr.py", line 322, in __call__
    fn(*args, **kw)
  File "/usr/lib/python3.8/site-packages/sqlalchemy/util/langhelpers.py", line 1513, in go
    return once_fn(*arg, **kw)
  File "/usr/lib/python3.8/site-packages/sqlalchemy/engine/strategies.py", line 199, in first_connect
    dialect.initialize(c)
  File "/usr/lib/python3.8/site-packages/sqlalchemy/dialects/postgresql/pg8000.py", line 215, in initialize
    super(PGDialect_pg8000, self).initialize(connection)
  File "/usr/lib/python3.8/site-packages/sqlalchemy/dialects/postgresql/base.py", line 2562, in initialize
    super(PGDialect, self).initialize(connection)
  File "/usr/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 312, in initialize
    self.server_version_info = self._get_server_version_info(
  File "/usr/lib/python3.8/site-packages/sqlalchemy/dialects/postgresql/base.py", line 2803, in _get_server_version_info
    v = connection.execute("select version()").scalar()
  File "/usr/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1003, in execute
    return self._execute_text(object_, multiparams, params)
  File "/usr/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1172, in _execute_text
    ret = self._execute_context(
  File "/usr/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1316, in _execute_context
    self._handle_dbapi_exception(
  File "/usr/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1514, in _handle_dbapi_exception
    util.raise_(exc_info[1], with_traceback=exc_info[2])
  File "/usr/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
    raise exception
  File "/usr/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1294, in _execute_context
    result = context._setup_crud_result_proxy()
  File "/usr/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 1243, in _setup_crud_result_proxy
    result = self.get_result_proxy()
  File "/usr/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 1218, in get_result_proxy
    return result.ResultProxy(self)
  File "/usr/lib/python3.8/site-packages/sqlalchemy/engine/result.py", line 775, in __init__
    self._init_metadata()
  File "/usr/lib/python3.8/site-packages/sqlalchemy/engine/result.py", line 807, in _init_metadata
    self._metadata = ResultMetaData(self, cursor_description)
  File "/usr/lib/python3.8/site-packages/sqlalchemy/engine/result.py", line 290, in __init__
    raw = self._merge_cursor_description(
  File "/usr/lib/python3.8/site-packages/sqlalchemy/engine/result.py", line 496, in _merge_cursor_description
    return [
  File "/usr/lib/python3.8/site-packages/sqlalchemy/engine/result.py", line 496, in <listcomp>
    return [
  File "/usr/lib/python3.8/site-packages/sqlalchemy/engine/result.py", line 616, in _merge_cols_by_none
    for (
  File "/usr/lib/python3.8/site-packages/sqlalchemy/engine/result.py", line 545, in _colnames_from_description
    colname = description_decoder(colname)
TypeError: expected bytes, str found

Versions.

• OS: Arch Linux
• Python: 3.8.6
• SQLAlchemy: 1.3.19, installed via pacman
• Database: PostgreSQL
• DBAPI: pg8000

[zzzeek]

yup this is due to a breaking change in pg8000 1.16.6, unfortunately. pg8000 is not officially supported until SQLAlchemy 1.4 where this issue has been fixed in b2f2bf1acc5799221981d30c2bf9e . you can use 1.16.5 for now to fix the issue for 1.3.

[zzzeek]

you can work around it this way also:

e = create_engine("postgresql+pg8000://...")
e.dialect.description_encoding = None

should work after that for 1.16.6

[marcoscesardev]

I had the same error, so i just changed in requirements
From:

SQLAlchemy>=1.3.4
pg8000>=1.12.5

To:

pg8000==1.12.5
SQLAlchemy==1.3.4